ElasticSearch(六)之java api基本用法

原创
2018/12/11 10:55
阅读数 741

 

1. Client 
说明:Client是Elasticsearch所有API的主入口,主要方法有: 

AdminClient admin() 获取ES管理客户端
GetRequestBuilder prepareGet() 准备一个GET请求
IndexRequestBuilder prepareIndex(String index, String type) 准备一个新增文档的请求
DeleteRequestBuilder prepareDelete() 准备一个删除文档的请求
BulkRequestBuilder prepareBulk() 准备一个批量操作的请求
SearchRequestBuilder prepareSearch(String... indices) 准备一个查询请求
UpdateRequestBuilder prepareUpdate() 准备一个更新的请求(更新的本质是先查询索引替换更新的值后进行替换,所以反而比插入更耗性能)


AdminClient 
说明:对ES进行管理的客户端,主要方法有: 

ClusterAdminClient cluster() 产生一个允许从集群中执行action或操作的client
IndicesAdminClient indices() 产生一个允许从索引中执行action或操作的client



IndicesAdminClient 
说明:对ES的index进行管理的客户端,主要方法有: 

IndicesExistsRequestBuilder prepareExists(String... indices) 准备一个判断索引是否存在的请求
TypesExistsRequestBuilder prepareTypesExists(String... index) 准备一个判断类型是否存在的请求
CreateIndexRequestBuilder prepareCreate(String index) 准备一个创建索引的请求
DeleteIndexRequestBuilder prepareDelete(String... indices) 准备一个删除索引的请求
AnalyzeRequestBuilder prepareAnalyze(@Nullable String index, String text) 准备一个对字符串进行分词的请求
PutIndexTemplateRequestBuilder preparePutTemplate(String name) 准备一个设置模板的请求
DeleteIndexTemplateRequestBuilder prepareDeleteTemplate(String name) 准备一个删除模板的请求
UpdateSettingsRequestBuilder prepareUpdateSettings(String... indices) 准备一个更新设置的请求,如更新副本数量等
PutMappingRequestBuilder preparePutMapping(String... indices) 准备一个新建映射关系的请求


QueryBuilders 
说明:为prepareSearch组装查询参数,如: 

Java代码 

  1. client.prepareSearch(esIndex).setTypes(esType).setQuery(QueryBuilders.matchQuery("global_ana_ch", "杭州西湖")).setFrom(0).setSize(1).get();  
matchAllQuery() 构造匹配所有文档的查询
matchQuery(String name, Object text) 构造查询一个被分析器分析过的字段的查询(match查询)
matchPhraseQuery(String name, Object text) 它和matchQuery的区别是它不会对传入的参数(text)进行分词,而是以其做为一个完整的词进行查询(类似百度查询时加引号的功能)如“有限公司”会分词成“有限”和“公司”,根据分词规则,“有限企业”、“大公司”这样的数据也会被查询出来,使用此方法后则必须包含“有限公司”才会被查询出
matchPhrasePrefixQuery(String name, Object text) 中文查询时matchPhraseQuery和matchPhrasePrefixQuery并没有什么区别,英文查询时matchPhrasePrefixQuery会以短语形式查询,查询时关键字不会被分词,而是直接以一个字符串的形式查询
commonTermsQuery(String name, Object text) 对query进行重写,区分低频词和高频词,并根据Elasticsearch传递的highFreqOccur和lowFreqOccur将高频词和低频词构造成BooleanQuery它的好处是减少了对高频词(如and)查询的性能影响,增加的查询效率
termQuery(String name, Object value) 多字段查询
termsQuery(String name, Object... values) 和termQuery类似,多个term组合
fuzzyQuery(String name, Object value) 模糊查询(like)
prefixQuery(String name, String prefix) 前缀匹配查询
rangeQuery(String name) 范围区间查询
wildcardQuery(String name, String query) 使用通配符查询(*,?)
regexpQuery(String name, String regexp) 正则查询org.apache.lucene.util.automaton.RegExp
queryStringQuery(String queryString) 字符串查询
boolQuery() 布尔型判断的查询must::多个查询条件的完全匹配,相当于 andmustNot::多个查询条件的相反匹配,相当于 notshould::至少有一个查询条件匹配, 相当于 or
SpanQuery SpanQuery是按照词在文章中的距离或者查询几个相邻词的查询。
spanFirstQuery 接受另一个跨度查询的匹配必须出现在第N的位置
spanNearQuery 接受多个跨度查询的匹配必须在指定的距离,并可能在相同的顺序
spanNotQuery 包装另一个跨度查询,排除了任何文档匹配查询
spanOrQuery 结合多个跨度查询,返回文档的匹配任何指定的查询
spanWithinQuery 和spanContainingQuery类似
spanContainingQuery 这个查询内部会有多个子查询,但是会设定某个子查询优先级更高,作用更大,通过关键字little和big来指定
spanMultiTermQueryBuilder 包装了 term, range, prefix, wildcard, regexp,或 fuzzy查询
spanTermQuery

和spanQuery类似,同时可以做为spanQuery的子句        

初始化elasticsearch客户端

   Settings settings = Settings.builder() 
                .put("cluster.name", "my-application")// 集群名称yml配置
                .put("client.transport.ping_timeout", "60s")// 超时时间
                .put("client.transport.sniff", false)//是否开启自动发现,非局域网关闭
                .put("client.transport.ignore_cluster_name", false).build();
        try {
            client = new PreBuiltTransportClient(settings).addTransportAddress(new TransportAddress(InetAddress.getByName(ip), 9300));
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }

elasticsearch常使用的用法

package com.xue;

 import com.alibaba.fastjson.JSON;
 import com.alibaba.fastjson.JSONObject;
 import org.apache.commons.io.FileUtils;
 import org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequestBuilder;
 import org.elasticsearch.action.get.GetResponse;
 import org.elasticsearch.action.search.SearchRequestBuilder;
 import org.elasticsearch.action.search.SearchResponse;
 import org.elasticsearch.client.Client;
import org.elasticsearch.client.IndicesAdminClient;
 import org.elasticsearch.index.query.BoolQueryBuilder;
 import org.elasticsearch.index.query.MatchQueryBuilder;
 import org.elasticsearch.index.query.QueryBuilders;
 import org.elasticsearch.search.aggregations.AggregationBuilders;
 import org.elasticsearch.search.aggregations.bucket.terms.StringTerms;
 import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder;
 import org.elasticsearch.search.sort.SortOrder;
 import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;

import java.io.File;
import java.io.IOException;
 import java.util.Map;

/**
 * Created by xuehan on 2018/12/10.
 */
/**1. index的操作*/
 public class ElSearch {
    private Client client = null;
    @Before
    public void init(){
        ESTest esTest = new ESTest();
        client = esTest.getClient();
    }
    /**2. mapping映射操作*/
     @Test
    public void testCreateIndex() {
        IndicesAdminClient indices = client.admin().indices();
        String esIndex = "myindex";
        // 判断inedx是否存在
        IndicesExistsResponse indicesExistsResponse = indices.prepareExists(esIndex).get();
        if (indicesExistsResponse.isExists()) {
            // 存在时先删除index
            indices.prepareDelete(esIndex).get();
        }
        // 创建前校验(index不存在)
        indicesExistsResponse = indices.prepareExists(esIndex).get();
        Assert.assertFalse(indicesExistsResponse.isExists());
        // 开始创建index
        indices.prepareCreate(esIndex).get();
        // 创建后校验(index存在)
        indicesExistsResponse = indices.prepareExists(esIndex).get();
        Assert.assertTrue(indicesExistsResponse.isExists());
    }
    @Test
    public void testMapping() throws IOException {
        String esIndex = "testMapping";
        IndicesAdminClient indices = client.admin().indices();
        // 判断inedx是否存在
        IndicesExistsResponse indicesExistsResponse = indices.prepareExists(esIndex).get();
        if (indicesExistsResponse.isExists()) {
            // 存在时先删除index
            indices.prepareDelete(esIndex).get();
        }
        // 创建新的index
        indices.prepareCreate(esIndex).get();
        // 执行前判断(mapping不存在)
        String esType = "esType";
        GetMappingsResponse getMappingsResponse = indices.prepareGetMappings(esIndex).setTypes(esType).get();
        Assert.assertTrue(getMappingsResponse.mappings().isEmpty());
        // 执行mapping
        PutMappingRequestBuilder builder = indices.preparePutMapping(esIndex).setType(esType);
        String mappingFile = getClass().getResource("/").getPath() + "search/lg_line_mapping.json";
        String mappingSource = FileUtils.readFileToString(new File(mappingFile));
        builder.setSource(mappingSource).get();
        // 执行后判断(mapping存在)
        getMappingsResponse = indices.prepareGetMappings(esIndex).setTypes(esType).get();
        Assert.assertFalse(getMappingsResponse.mappings().isEmpty());
    }
    @Test
    public void testIndexing() {
        String esType = "esType";
        String esIndex = "myindex";
        // 不管有没有,先删除数据
        client.prepareDelete(esIndex, esType, "1").execute();

        // 执行前判断(数据不存在)
        GetResponse response = client.prepareGet(esIndex, esType, "1").get();
        Assert.assertNull(response.getSourceAsString());

        // 插入数据
        LineIndexingVO lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(1L);
        lineIndexingVO.setLineNo("LM2017041913250001");
        lineIndexingVO.setGlobal_ana_ch("浙江,杭州,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map = JSONObject.parseObject(lineIndexingVOStr, Map.class);
        client.prepareIndex(esIndex, esType, "1").setSource(map).execute();

        // 执行后判断(数据存在)
        response = client.prepareGet(esIndex, esType, "1").get();
        Assert.assertNotNull(response.getSourceAsString());
    }
    /**4. matchQuery查询**/
    /**
     * 关键词匹配,global_ana_ch是一个分词的字段可以进行搜索匹配
     */
    @Test
    public void testMatchQuery() {
        String esType = "esType";
        String esIndex = "myindex";
         // 插入数据
        LineIndexingVO lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(1L);
        lineIndexingVO.setLineNo("LM2017041913250001");
        lineIndexingVO.setGlobal_ana_ch("浙江,杭州,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map = JSONObject.parseObject(lineIndexingVOStr, Map.class);
        client.prepareIndex(esIndex, esType, "1").setSource(map).execute();

        // 执行matchQuery查询
        SearchRequestBuilder searchRequestBuilder = client.prepareSearch(esIndex).setTypes(esType);
        MatchQueryBuilder matchQuery = QueryBuilders.matchQuery("global_ana_ch", "杭州西湖");
        SearchResponse response = searchRequestBuilder.setQuery(matchQuery).setFrom(0).setSize(1).get();
        Assert.assertEquals(1, response.getHits().getTotalHits());
    }
    /**
     * 过滤查询
     */
    @Test
    public void testTerm() {
        String esType = "esType";
        String esIndex = "myindex";
        // 插入数据
        LineIndexingVO lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(1L);
        lineIndexingVO.setLineNo("LM2017041913250001");
        lineIndexingVO.setGlobal_ana_ch("浙江,杭州,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map = JSONObject.parseObject(lineIndexingVOStr, Map.class);
        client.prepareIndex(esIndex, esType, "1").setSource(map).execute();

        // 执行term查询,相当于select * from lg_line where id=1 and lineNo="LM2017041913250001"
        SearchRequestBuilder searchRequestBuilder = client.prepareSearch(esIndex).setTypes(esType);
         BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
        boolQueryBuilder.must(QueryBuilders.termQuery("lineNo", "LM2017041913250001"));
       // boolQueryBuilder.must(QueryBuilders.termQuery("id", 1));
        SearchResponse response = searchRequestBuilder.setQuery(boolQueryBuilder).get();
        Assert.assertEquals(1, response.getHits().getTotalHits());

        // 执行term查询,相当于select * from lg_line where id=1 and lineNo="LM2017041913250001" and lineNo!="LM2017041913250001"
        boolQueryBuilder.mustNot(QueryBuilders.termQuery("lineNo", "LM2017041913250001"));
        response = searchRequestBuilder.setQuery(boolQueryBuilder).setFrom(0).setSize(1).get();
        Assert.assertEquals(0, response.getHits().getTotalHits());
    }
    /**
     * 排序测试
     */
    @Test
    public void testOrder() {
        String esType = "esType";
        String esIndex = "myindex";
         // 插入数据
        LineIndexingVO lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(1L);
        lineIndexingVO.setLineNo("LM2017041913250001");
        lineIndexingVO.setGlobal_ana_ch("浙江,杭州,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map = JSONObject.parseObject(lineIndexingVOStr, Map.class);

        client.prepareIndex(esIndex, esType, "1").setSource(map).execute();
        // 第二条数据比第一条少了“杭州”
        lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(2L);
        lineIndexingVO.setLineNo("LM2017041913250002");
        lineIndexingVO.setGlobal_ana_ch("浙江,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr1 = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map1 = JSONObject.parseObject(lineIndexingVOStr1, Map.class);

        client.prepareIndex(esIndex, esType, "2").setSource(map1).execute();

        MatchQueryBuilder matchQuery = QueryBuilders.matchQuery("global_ana_ch", "浙江杭州");
        // 根据匹配度倒序排列,匹配度高的排在前面
        SearchResponse response = client.prepareSearch(esIndex).setTypes(esType).setQuery(matchQuery).setFrom(0).setSize(2)
                .addSort("_score", SortOrder.DESC).get();
        Assert.assertEquals("1", response.getHits().iterator().next().getId());
        // 根据匹配度顺序排列,匹配度低的排在前面
        response = client.prepareSearch(esIndex).setTypes(esType).setQuery(matchQuery).setFrom(0).setSize(2).addSort("_score", SortOrder.ASC).get();
        Assert.assertEquals("2", response.getHits().iterator().next().getId());
    }
    /**7.统计查询**/
     @Test
    public void testAggs() {
        String esType = "esType1";
        String esIndex = "aggs";
         // 插入数据
        LineIndexingVO lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(1L);
        lineIndexingVO.setLineType("1");
        lineIndexingVO.setGlobal_ana_ch("浙江,杭州,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr1 = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map1 = JSONObject.parseObject(lineIndexingVOStr1, Map.class);
        client.prepareIndex(esIndex, esType, "1").setSource(map1).execute();

        lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(2L);
        lineIndexingVO.setLineType("1");
        lineIndexingVO.setGlobal_ana_ch("浙江,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr2 = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map2 = JSONObject.parseObject(lineIndexingVOStr2, Map.class);
        client.prepareIndex(esIndex, esType, "2").setSource(map2).execute();

        lineIndexingVO = new LineIndexingVO();
        lineIndexingVO.setId(3L);
        lineIndexingVO.setLineType("2");
        lineIndexingVO.setGlobal_ana_ch("浙江,冬瓜,西瓜,地铁,番茄泡");
        String lineIndexingVOStr3 = JSONObject.toJSONString(lineIndexingVO);
        Map<String, Object> map3 = JSONObject.parseObject(lineIndexingVOStr3, Map.class);
        client.prepareIndex(esIndex, esType, "3").setSource(map3).execute();

         /**
          * 需要设置索引
          * {
          "properties": {
          "lineType": {
          "type":     "text",
          "fielddata": true
          }
          }
          }
          http://47.105.74.94:9200/aggs/_mapping/esType1
          */
        // 根据lineType进行分类统计,type=1的有2条,type=2的有1条
        TermsAggregationBuilder termsBuilder = AggregationBuilders.terms("by_lineType").field("lineType");
        SearchResponse response = client.prepareSearch(esIndex).setTypes(esType).addAggregation(termsBuilder).setSize(10).get();
        StringTerms aggregation = response.getAggregations().get("by_lineType");
        Assert.assertEquals(2, aggregation.getBucketByKey("1").getDocCount());
        Assert.assertEquals(1, aggregation.getBucketByKey("2").getDocCount());
    }
    class LineIndexingVO{
         private double id;
        private String lineNo;
        private String lineType;
        private String global_ana_ch;

        public double getId() {
            return id;
        }

        public void setId(double id) {
            this.id = id;
        }

        public String getLineNo() {
            return lineNo;
        }

        public void setLineNo(String lineNo) {
            this.lineNo = lineNo;
        }

        public String getGlobal_ana_ch() {
            return global_ana_ch;
        }

        public void setGlobal_ana_ch(String global_ana_ch) {
            this.global_ana_ch = global_ana_ch;
        }

        public String getLineType() {
            return lineType;
        }

        public void setLineType(String lineType) {
            this.lineType = lineType;
        }
    }
}

 

展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部