文档章节

Spring Batch_Multi-threaded Step

秋风醉了
 秋风醉了
发布于 2014/11/13 19:11
字数 505
阅读 1559
收藏 1

Spring Batch_Multi-threaded Step

spring官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html

The simplest way to start parallel processing is to add a TaskExecutor to your Step configuration, e.g. as an attribute of the tasklet:

<step id="loading">
    <tasklet task-executor="taskExecutor">...</tasklet>
</step>

In this example the taskExecutor is a reference to another bean definition, implementing the TaskExecutor interface. TaskExecutor is a standard Spring interface, so consult(查阅,参照) the Spring User Guide for details of available implementations. The simplest multi-threaded TaskExecutor is a SimpleAsyncTaskExecutor.

 

The result of the above configuration will be that the Step executes by reading, processing and writing each chunk of items (each commit interval) in a separate thread of execution. Note that this means there is no fixed order for the items to be processed, and a chunk might contain items that are non-consecutive(不连续的) compared to the single-threaded case. In addition to any limits placed by the task executor (e.g. if it is backed by a thread pool), there is a throttle limit in the tasklet configuration which defaults to 4. You may need to increase this to ensure that a thread pool is fully utilised, e.g.

<step id="loading">
	<tasklet task-executor="taskExecutor" throttle-limit="20">...</tasklet>
</step>

Note also that there may be limits placed on concurrency by any pooled resources used in your step, such as a DataSource. Be sure to make the pool in those resources at least as large as the desired number of concurrent threads in the step.

 

There are some practical limitations of using multi-threaded Steps for some common Batch use cases. Many participants in a Step (e.g. readers and writers) are stateful(有状态的), and if the state is not segregated(分离,隔开的) by thread, then those components are not usable in a multi-threaded Step. In particular most of the off-the-shelf(现成的) readers and writers from Spring Batch are not designed for multi-threaded use. It is, however, possible to work with stateless or thread safe readers and writers, and there is a sample (parallelJob) in the Spring Batch Samples that show the use of a process indicator (see Section 6.12, “Preventing State Persistence”) to keep track of items that have been processed in a database input table.

 

Spring Batch provides some implementations of ItemWriter and ItemReader. Usually they say in the Javadocs if they are thread safe or not, or what you have to do to avoid problems in a concurrent environment. If there is no information in Javadocs, you can check the implementation to see if there is any state. If a reader is not thread safe, it may still be efficient(有效率的) to use it in your own synchronizing delegator. You can synchronize the call to read() and as long as the processing and writing is the most expensive part of the chunk your step may still complete much faster than in a single threaded configuration.

=================END=================

© 著作权归作者所有

共有 人打赏支持
秋风醉了
粉丝 241
博文 566
码字总数 417505
作品 0
朝阳
程序员
私信 提问
选择正确的 JSR-352 (Java批处理)实现的 10 个关键点

您可能已经知道,JSR-352是一个Java批处理的标准。该标准在今年发布,并被包含在了JEE7之中,这意味着,所有的JEE7应用服务器都会有批处理的能力。正如我在上一篇文章中指出,即使您的应用服...

oschina
2013/12/07
7.8K
4
Spring Batch_Configuring a Step for Restart

Spring Batch_Configuring a Step for Restart spring官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html#stepRestart 当一个普通的 job 处于complete ......

秋风醉了
2014/11/14
0
0
Spring Batch 之 背景框架简介

版权声明: https://blog.csdn.net/wuzhiwei549/article/details/85392128 Spring Batch 是一个轻量级的、全面的批处理框架,用于开发对企业系统的日常操作至关重要的健壮批处理应用程序。S...

91Vincent
2018/12/30
0
0
Spring Batch 之 Sample(Hello World)(三)

通过前面两篇关于Spring Batch文章的介绍,大家应该已经对Spring Batch有个初步的概念了。这篇文章,将通过一个”Hello World!”实例,和大家一起探讨关于Spring Batch的一些基本配置和实现...

长平狐
2012/08/27
807
0
Spring Batch 之 Job的创建和调用

版权声明: https://blog.csdn.net/wuzhiwei549/article/details/85394406 在上一篇文章 Spring Batch 之 背景框架简介 中,已经概述了Batch的基本架构组织,并且运行了简易demo。 在接下来的...

91Vincent
2018/12/30
0
0

没有更多内容

加载失败,请刷新页面

加载更多

centos7重置密码、单用户模式、救援模式、ls命令、chmod命令

在工作当中如果我们错误的配置了文件使服务器不能正常启动或者忘记密码不能登录系统,如何解决这些问题呢?重装系统是可以实现的,但是往往不能轻易重装系统的,下面用忘记密码作为例子讲解如...

李超小牛子
今天
3
0
Python如何开发桌面应用程序?Python基础教程,第十三讲,图形界面

当使用桌面应用程序的时候,有没有那么一瞬间,想学习一下桌面应用程序开发?行业内专业的桌面应用程序开发一般是C++,C#来做,Java开发的也有,但是比较少。本节课会介绍Python的GUI(图形用...

程序员补给栈
今天
5
0
kafka在的使用

一、基本概念 介绍 Kafka是一个分布式的、可分区的、可复制的消息系统。它提供了普通消息系统的功能,但具有自己独特的设计。 这个独特的设计是什么样的呢? 首先让我们看几个基本的消息系统...

狼王黄师傅
今天
3
0
Android JNI总结

0x01 JNI介绍 JNI是Java Native Interface的缩写,JNI不是Android专有的东西,它是从Java继承而来,但是在Android中,JNI的作用和重要性大大增强。 JNI在Android中起着连接Java和C/C++层的作...

天王盖地虎626
昨天
3
0
大数据教程(11.8)Hive1.2.2简介&初体验

上一篇文章分析了Hive1.2.2的安装,本节博主将分享Hive的体验&Hive服务端和客户端的使用方法。 一、Hive与hadoop直接的关系 Hive利用HDFS存储数据,利用MapReduce查询数据。 二、Hive与传统数...

em_aaron
昨天
4
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部