文档章节

Spring Batch_Multi-threaded Step

秋风醉了
 秋风醉了
发布于 2014/11/13 19:11
字数 505
阅读 1553
收藏 1

Spring Batch_Multi-threaded Step

spring官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html

The simplest way to start parallel processing is to add a TaskExecutor to your Step configuration, e.g. as an attribute of the tasklet:

<step id="loading">
    <tasklet task-executor="taskExecutor">...</tasklet>
</step>

In this example the taskExecutor is a reference to another bean definition, implementing the TaskExecutor interface. TaskExecutor is a standard Spring interface, so consult(查阅,参照) the Spring User Guide for details of available implementations. The simplest multi-threaded TaskExecutor is a SimpleAsyncTaskExecutor.

 

The result of the above configuration will be that the Step executes by reading, processing and writing each chunk of items (each commit interval) in a separate thread of execution. Note that this means there is no fixed order for the items to be processed, and a chunk might contain items that are non-consecutive(不连续的) compared to the single-threaded case. In addition to any limits placed by the task executor (e.g. if it is backed by a thread pool), there is a throttle limit in the tasklet configuration which defaults to 4. You may need to increase this to ensure that a thread pool is fully utilised, e.g.

<step id="loading">
	<tasklet task-executor="taskExecutor" throttle-limit="20">...</tasklet>
</step>

Note also that there may be limits placed on concurrency by any pooled resources used in your step, such as a DataSource. Be sure to make the pool in those resources at least as large as the desired number of concurrent threads in the step.

 

There are some practical limitations of using multi-threaded Steps for some common Batch use cases. Many participants in a Step (e.g. readers and writers) are stateful(有状态的), and if the state is not segregated(分离,隔开的) by thread, then those components are not usable in a multi-threaded Step. In particular most of the off-the-shelf(现成的) readers and writers from Spring Batch are not designed for multi-threaded use. It is, however, possible to work with stateless or thread safe readers and writers, and there is a sample (parallelJob) in the Spring Batch Samples that show the use of a process indicator (see Section 6.12, “Preventing State Persistence”) to keep track of items that have been processed in a database input table.

 

Spring Batch provides some implementations of ItemWriter and ItemReader. Usually they say in the Javadocs if they are thread safe or not, or what you have to do to avoid problems in a concurrent environment. If there is no information in Javadocs, you can check the implementation to see if there is any state. If a reader is not thread safe, it may still be efficient(有效率的) to use it in your own synchronizing delegator. You can synchronize the call to read() and as long as the processing and writing is the most expensive part of the chunk your step may still complete much faster than in a single threaded configuration.

=================END=================

© 著作权归作者所有

共有 人打赏支持
秋风醉了
粉丝 239
博文 573
码字总数 417084
作品 0
朝阳
程序员
私信 提问
Spring Batch_Configuring a Step for Restart

Spring Batch_Configuring a Step for Restart spring官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html#stepRestart 当一个普通的 job 处于complete ......

秋风醉了
2014/11/14
0
0
Spring Batch_JOB重启机制

Spring Batch_JOB重启机制 在这一篇文章 对于restart做了试验,http://my.oschina.net/xinxingegeya/blog/344817在这片文章里,我们只是当job成功时,重启了job,对于job失败后,重启job有...

秋风醉了
2014/11/19
0
0
Spring Batch_Parallel Steps

Spring BatchParallel Steps使用并行的Step spring 官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html#scalabilityParallelSteps As long as the applic......

秋风醉了
2014/11/14
0
2
Spring Batch_Step Flow

Spring Batch学习_Step Flow http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html With the ability to group steps together within an owning job comes the nee......

秋风醉了
2014/10/31
0
0
Spring Batch_实现Tasklet

Spring Batch_实现Tasklet Chunk-oriented processing is not the only way to process in a Step. What if a Step must consist as a simple stored procedure call? You could implement ......

秋风醉了
2014/12/01
0
0

没有更多内容

加载失败,请刷新页面

加载更多

vue-router的HTML5 History 模式设置

VUE是当下最火爆的前端框架之一,vue-router是vue项目中几乎都会用到的组件,然而体验一时爽,其实坑不少。本篇经验将详细介绍vue-router的两种mode效果和开发测试环境下的问题,并给出解决方...

peakedness丶
29分钟前
1
0
Dubbo 源码分析 - 服务引用

1. 简介 在上一篇文章中,我详细的分析了服务导出的原理。本篇文章我们趁热打铁,继续分析服务引用的原理。在 Dubbo 中,我们可以通过两种方式引用远程服务。第一种是使用服务直联的方式引用...

小刀爱编程
39分钟前
1
0
redis_集群原理

数据分布 数据分布理论 分布式系统数据分区规则:常见的有哈希分区和顺序分区两种。 哈希分区 常见的hash分区规则有:节点取余分区,一致性hash分区,虚拟槽分区. redis数据分区 redis集群功能...

grace_233
39分钟前
0
0
数据库(mysql)业务用户表批量造测试用户密码数据[方案]

博主这几天在配合公司测试做商城商品秒杀场景的压测,由于秒杀需要登录,所以需要为测试创建多个用户密码。此项任务可能在今后还会遇到,先做个笔录。同时,让小伙伴们需要的时候也可以有个参...

em_aaron
55分钟前
2
0
JDK_jvisualvm访问远程weblogic服务

1.首先需要在远程的weblogic的域下面,找到/bin/ setDomainEnv.sh ,需要在此文件下加入如下内容: -Dcom.sun.management.jmxremote.port=1090 ##端口号可以自己指定 -Dcom.sun.management.j...

四年级小学生
今天
1
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部