spring-session之4 redis集群配置

原创
2016/10/14 12:07
阅读数 3.4K

spring-session之4 redis集群配置

前文,我们 spring-session之3 redis配置参数配置, 我们可以配置简单的ip和port,但是生产环境,我们的redis是做了集群,肯定不是单点,此时就不能单单hostName 和 port了, 怎么办?

好,今天的目标是:

  1. 配置spring-session redis 集群

1. 我们原来的 redis data 配置

1.1. spring-redis.xml

	
	<?xml version="1.0" encoding="UTF-8"?>
	<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
		xmlns:context="http://www.springframework.org/schema/context" xmlns:loxia="http://loxia.benjamin.cn/schema/core"
		xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
		xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
			http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
			http://loxia.benjamin.cn/schema/core http://loxia.benjamin.cn/schema/core/loxia-spring-ext.xsd
			http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd	
			">
		
		<bean id="redisconfig" class="redis.clients.jedis.JedisPoolConfig">
			<property  name="maxActive" >
				<value type="long">#{redis['redis.MaxActive']}</value>
			</property>
			<property  name="maxIdle" >
				<value type="long">#{redis['redis.MaxIdle']}</value>
			</property>
			<property  name="maxWait">
				<value type="long">#{redis['redis.MaxWait']}</value>
			</property>
			<property  name="testOnBorrow" >
				<value type="boolean">true</value>
			</property>
			<property  name="testOnReturn" >
				<value type="boolean">true</value>
			</property>
		</bean>
		
		<bean id="jedisSentinelPool" class="redis.clients.jedis.JedisSentinelPool">
			<constructor-arg ref="redisconfig"></constructor-arg>
				 
			<constructor-arg type="java.util.Set">
				<set>
					<value>#{redis['redis.sentinel1']}</value>
					<value>#{redis['redis.sentinel2']}</value>
					<value>#{redis['redis.sentinel3']}</value>
				</set>
				
			</constructor-arg>
	 			
			<constructor-arg >
				<value>#{redis['redis.mastername']}</value>
			</constructor-arg>
			
		</bean>
	</beans>

1.2 redis.properties


	#A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
	redis.sentinel1=redis01.public.test.baozun.cn:26379
	redis.sentinel2=redis02.public.test.baozun.cn:26379
	redis.sentinel3=redis03.public.test.baozun.cn:26379

	redis.mastername=testenv
	
	redis.timeout=1000
	redis.MaxActive=50
	redis.MaxIdle=60000
	redis.MaxWait=5000
	
	##这个是我们自定义的 key前缀, 以便用同一套redis集群,适应多个不同的商城
	redis.keystart=feilongdev

2.redis 集群配置

那么我们怎么使用上面的环境来配置我们的spring-session呢?

使用通过上述的参数表格, JedisPoolConfig 可以使用,但是没有 JedisSentinelPool参数

JedisShardInfo属性并不是list,只是 包含了jedis服务器的一些信息 ,咋整?

这时候,我们来看看 JedisConnectionFactory 构造函数:

可以看出 ,构造函数支持 RedisSentinelConfiguration 或者 RedisClusterConfiguration

此处的 JedisConnectionFactory 是 spring-data-redis 内容, 有兴趣的可以看看, 这里也有入门版的教程,参见使用Spring Data Redis操作Redis(一)

2.1 RedisSentinelConfiguration

我们先使用 redis 哨兵机制的集群

那么我们尝试修改以下配置配置:


	<?xml version="1.0" encoding="UTF-8"?>
	<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
		xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
		xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
			http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
			http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd	
			">
	
		<context:annotation-config />
	
		<util:properties id="redis" location="classpath:redis.properties"></util:properties>
	
		<!-- RedisHttpSessionConfiguration -->
		<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />
	
		<!--JedisConnectionFactory -->
		<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
			<constructor-arg index="0">
				<bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
					<constructor-arg index="0" value="#{redis['redis.mastername']}" />
					<constructor-arg index="1">
						<set>
							<value>#{redis['redis.sentinel1']}</value>
							<value>#{redis['redis.sentinel2']}</value>
							<value>#{redis['redis.sentinel3']}</value>
						</set>
					</constructor-arg>
				</bean>
			</constructor-arg>
	
			<constructor-arg index="1">
				<bean class="redis.clients.jedis.JedisPoolConfig">
					<property name="maxActive" value="#{redis['redis.MaxActive']}" />
					<property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
					<property name="maxWait" value="#{redis['redis.MaxWait']}" />
					<property name="testOnBorrow" value="true" />
					<property name="testOnReturn" value="true" />
				</bean>
			</constructor-arg>
		</bean>
	
	</beans>

启动应用

出现以下的错误


Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'maxActive' of bean class [redis.clients.jedis.JedisPoolConfig]: Bean property 'maxActive' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
	at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:1044)
	at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:904)
	at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:75)
	at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:57)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1452)
	... 30 more

JedisPoolConfig 不支持 maxActive 属性了 see https://github.com/xetorthio/jedis/issues/1031

好吧, 修改下配置 spring-session.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd	
		">

	<context:annotation-config />

	<util:properties id="redis" location="classpath:redis.properties"></util:properties>

	<!-- RedisHttpSessionConfiguration -->
	<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

	<!--JedisConnectionFactory -->
	<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
		<constructor-arg index="0">
			<bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
				<constructor-arg index="0" value="#{redis['redis.mastername']}" />
				<constructor-arg index="1">
					<set>
						<value>#{redis['redis.sentinel1']}</value>
						<value>#{redis['redis.sentinel2']}</value>
						<value>#{redis['redis.sentinel3']}</value>
					</set>
				</constructor-arg>
			</bean>
		</constructor-arg>

		<constructor-arg index="1">
			<bean class="redis.clients.jedis.JedisPoolConfig">
				<property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
				<property name="testOnBorrow" value="true" />
				<property name="testOnReturn" value="true" />

				<!-- 新版jedis 不支持这个参数了 -->
				<!-- <property name="maxWait" value="#{redis['redis.MaxWait']}" /> -->
				<!-- <property name="maxActive" value="#{redis['redis.MaxActive']}" /> -->
			</bean>
		</constructor-arg>
	</bean>

</beans>

redis.properties

	
	#A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
	redis.sentinel1=redis01.public.test.baozun.cn:26379
	redis.sentinel2=redis02.public.test.baozun.cn:26379
	redis.sentinel3=redis03.public.test.baozun.cn:26379
	
	redis.mastername=testenv
	
	redis.timeout=1000
	
	redis.MaxIdle=60000
	
	#新版jedis 不支持这个参数了
	#redis.MaxActive=50
	#redis.MaxWait=5000
	
	redis.keystart=gowilddev

启动

报了新错


	Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR Unsupported CONFIG parameter: notify-keyspace-events
		at redis.clients.jedis.Protocol.processError(Protocol.java:117)
		at redis.clients.jedis.Protocol.process(Protocol.java:151)
		at redis.clients.jedis.Protocol.read(Protocol.java:205)
		at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
		at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:196)
		at redis.clients.jedis.Jedis.configSet(Jedis.java:2612)
		at org.springframework.data.redis.connection.jedis.JedisConnection.setConfig(JedisConnection.java:633)
		... 25 more

什么鬼,参见 ERR Unsupported CONFIG parameter: notify-keyspace-events

看来我是使用新的jedis 连接了老版本的redis集群

服务器上面的 redis 版本

找运维组重新搭建 redis新版本的集群环境

[vmuser@test01 ~]$ redis-cli --version
redis-cli 3.2.4

由于采用了新版的集群技术,所以我们只能使用 RedisClusterConfiguration

2.2 RedisClusterConfiguration

2.2.1 3主3从 Cluster服务端配置

我们先看看运维组兄弟配置的集群情况

[vmuser@test01 ~]$ /home/vmuser/redis-3.2.4/src/redis-trib.rb check 10.88.21.31:10000

可以看到是官网推荐的标准的 3主 3从 方案,参见 官方文档 http://redisdoc.com/topic/cluster-tutorial.html

master slave
10.88.21.31:10000 10.88.22.25:10001
10.88.22.25:10000 10.88.21.31:10002
10.88.21.31:10001 10.88.22.25:10002

2.2.2 spring配置

这时我们可以使用 org.springframework.data.redis.connection.RedisClusterConfiguration ,注意 since spring-data-redis 1.7


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd	
		">

	<context:annotation-config />

	<util:properties id="redis" location="classpath:redis-cluster.properties"></util:properties>

	<!-- RedisHttpSessionConfiguration -->
	<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

	<!--JedisConnectionFactory -->
	<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
		<constructor-arg index="0">
			<bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
				<constructor-arg index="0">
					<set>
						<!-- slave -->
						<!-- <value>10.88.22.25:10001</value>
							<value>10.88.21.31:10002</value>
							<value>10.88.22.25:10002</value> -->

						<!-- master -->
						<value>10.88.21.31:10000</value>
						<value>10.88.22.25:10000</value>
						<value>10.88.21.31:10001</value>
					</set>
				</constructor-arg>

				<!-- 一般当此值设置过大时,容易报:Too many Cluster redirections -->
				<!-- <property name="maxRedirects">3</property> -->
			</bean>
		</constructor-arg>

		<constructor-arg index="1">
			<bean class="redis.clients.jedis.JedisPoolConfig">
				<property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
				<property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
				<property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />

				<!-- 新版jedis 不支持这个参数了 -->
				<!-- <property name="maxWait" value="#{redis['redis.jedisPoolConfig.MaxWait']}" /> -->
				<!-- <property name="maxActive" value="#{redis['redis.jedisPoolConfig.MaxActive']}" /> -->
			</bean>
		</constructor-arg>
	</bean>

</beans>

此时,我们仅需要配置 master 到 RedisClusterConfiguration 中,系统会自动分配

上面配置可以提取精炼到配置文件中

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
		http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd	
		">

	<context:annotation-config />

	<util:properties id="redis" location="classpath:redis-cluster.properties"></util:properties>

	<!-- RedisHttpSessionConfiguration -->
	<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />

	<!--JedisConnectionFactory -->
	<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
		<constructor-arg index="0">
			<!-- since spring-data-redis 1.7 -->
			<bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
				<constructor-arg index="0">
					<set>
						<value>#{redis['redis.redisClusterConfiguration.clusters']}</value>
					</set>
				</constructor-arg>

				<!--
					用于 redis.clients.jedis.JedisCluster.JedisCluster(Set<HostAndPort>, int, int, GenericObjectPoolConfig) 第三个参数 maxRedirections
					默认值是5
					一般当此值设置过大时,容易报:Too many Cluster redirections
				-->
				<property name="maxRedirects" value="#{redis['redis.redisClusterConfiguration.maxRedirects']}" />
			</bean>
		</constructor-arg>

		<constructor-arg index="1">
			<bean class="redis.clients.jedis.JedisPoolConfig">
				<property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
				<property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
				<property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />

				<!-- 新版jedis 不支持这个参数了 -->
				<!-- <property name="maxWait" value="#{redis['redis.jedisPoolConfig.MaxWait']}" /> -->
				<!-- <property name="maxActive" value="#{redis['redis.jedisPoolConfig.MaxActive']}" /> -->
			</bean>
		</constructor-arg>
	</bean>

</beans>

redis-cluster.properties


#############for org.springframework.data.redis.connection.RedisClusterConfiguration###################
#只需要配置 master
#理论上只需要配置一个节点即可,配置多个是为了防止单个节点挂掉,
redis.redisClusterConfiguration.clusters=10.88.21.31:10000,10.88.22.25:10000,10.88.21.31:10001

#用于 redis.clients.jedis.JedisCluster.JedisCluster(Set<HostAndPort>, int, int, GenericObjectPoolConfig) 第三个参数 maxRedirections
#默认值是5
#一般当此值设置过大时,容易报:Too many Cluster redirections
redis.redisClusterConfiguration.maxRedirects=3

###########for redis.clients.jedis.JedisPoolConfig##############################

redis.jedisPoolConfig.MaxIdle=60000
redis.jedisPoolConfig.testOnBorrow=true
redis.jedisPoolConfig.testOnReturn=true

#新版jedis 不支持这个参数了
#redis.jedisPoolConfig.MaxActive=50
#redis.jedisPoolConfig.MaxWait=5000

#redis.keystart=gowilddev

启动

成功没有报错

2.2.3 为什么标准的 3主 3从 方案,我们 clusterNodes 仅配置了 3个 master?

  1. 首先配置多个是为了防止单个节点挂掉,理论上只需要配置一个节点即可
  2. 至于为什么不配置slave节点,参见 https://github.com/spring-projects/spring-data-examples spring data 示例中也只配置了master

2.2.4 关于 RedisClusterConfiguration 中的 maxRedirects 属性

重试次数,在执行失败后,进行的重试次数,默认是5, 参见 https://www.zybuluo.com/SailorXiao/note/159072#解读

主要用于 org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(RedisClusterConfiguration, GenericObjectPoolConfig)

构造 redis.clients.jedis.JedisCluster.JedisCluster(Set<HostAndPort>, int, int, GenericObjectPoolConfig) 使用的

源码:


	/**
	 * Creates {@link JedisCluster} for given {@link RedisClusterConfiguration} and {@link GenericObjectPoolConfig}.
	 * 
	 * @param clusterConfig must not be {@literal null}.
	 * @param poolConfig can be {@literal null}.
	 * @return
	 * @since 1.7
	 */
	protected JedisCluster createCluster(RedisClusterConfiguration clusterConfig, GenericObjectPoolConfig poolConfig) {

		Assert.notNull("Cluster configuration must not be null!");

		Set<HostAndPort> hostAndPort = new HashSet<HostAndPort>();
		for (RedisNode node : clusterConfig.getClusterNodes()) {
			hostAndPort.add(new HostAndPort(node.getHost(), node.getPort()));
		}

		int redirects = clusterConfig.getMaxRedirects() != null ? clusterConfig.getMaxRedirects().intValue() : 5;

		if (StringUtils.hasText(getPassword())) {
			throw new IllegalArgumentException("Jedis does not support password protected Redis Cluster configurations!");
		}

		if (poolConfig != null) {
			return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
		}
		return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
	}

一般当此值设置过大时,容易报:Too many Cluster redirections

3.问题

3.1 spring-session>1.2.2.RELEASE 不兼容 spring-data-redis > 1.7.4.RELEASE

目前 spring-session 最新版本是 1.2.2.RELEASE,内置依赖 spring-data-redis 版本是 1.7.1.RELEASE 而目前 spring-data-redis 最新版本是 1.7.4.RELEASE

尝试升级 spring-data-redis>1.7.4.RELEASE 启动报异常


Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionRedisTemplate' defined in class path resource [org/springframework/session/data/redis/config/annotation/web/http/RedisHttpSessionConfiguration.class]: Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.<init>(Ljava/lang/ClassLoader;)V
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:921)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:864)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:779)
	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:817)
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:745)
	... 38 more
Caused by: java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.<init>(Ljava/lang/ClassLoader;)V
	at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.<init>(JdkSerializationRedisSerializer.java:53)
	at org.springframework.data.redis.core.RedisTemplate.afterPropertiesSet(RedisTemplate.java:119)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
	... 49 more

4.参考

--to be continued

展开阅读全文
打赏
0
9 收藏
分享
加载中
更多评论
打赏
0 评论
9 收藏
0
分享
返回顶部
顶部