Java客户端并发更新Elastic索引总结
如开始有一个index
{"count":0}
10个并发线程对count进行加一操作,期待结果为10.代码如下:
// create index
client.prepareIndex(index, type, id)
.setSource(XContentFactory.jsonBuilder().startObject().field("count", 0).endObject())
.execute().get();
// multi thread update
int count = 10;
CountDownLatch latch = new CountDownLatch(count);
ExecutorService service = Executors.newFixedThreadPool(10);
for (int i = 0; i < count; i++) {
service.execute(new Runnable() {
@Override
public void run() {
try {
client.prepareUpdate(index, type, id)
.setScript("ctx._source.count += 1", ScriptService.ScriptType.INLINE).execute();
} catch (Exception e) {
System.err.println(Thread.currentThread().getName());
e.printStackTrace();
} finally {
latch.countDown();
}
}
});
}
// check result
latch.await();
service.shutdown();
GetResponse getResponse = client.prepareGet(index, type, id).execute().get();
System.out.println(getResponse.getSourceAsString());
但实际结果会小于10, 不知丢失的update去哪儿了?
通过Wireshark抓包工具分析,实际也出现了更新冲突,但程序不调用get的话就不会抛出此异常,抓包截图如下所示:
若将上述update索引语句修改为:
client.prepareUpdate(index, type, id)
.setScript("ctx._source.count += 1", ScriptService.ScriptType.INLINE).execute()
.get();
即在最后调用了get。这时会报错:
Caused by: org.elasticsearch.index.engine.VersionConflictEngineException: [temp-index][2] [temp-type][1]: version conflict, current [2], provided [1]
at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:432)
at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:375)
......
经尝试,发现若想最后结果为10,需要满足如下两个条件,
//1. 单线程
ExecutorService service = Executors.newFixedThreadPool(1);
//2. 显式调用get
client.prepareUpdate(index, type, id)
.setScript("ctx._source.count += 1", ScriptService.ScriptType.INLINE).execute()
.get();
补充:
其实连续顺序执行也会出现结果不正确的情况,
for (int i = 0; i < 10; i++) {
client.prepareUpdate(index, type, id)
.setScript("ctx._source.count += 1", ScriptService.ScriptType.INLINE).execute();
}
除非显式调用get. 因为通过Shell脚本并发执行update也能出现update conflict,
for ((i=0; i<10; i++))
do
curl -XPOST 'localhost:9200/test/type/1/_update?pretty' -d '{
"script" : "ctx._source.count += 1"
}' &
done
故可以认为是服务端的问题。