JPA项目多数据源模式整合Sharding-jdbc实现数据脱敏

原创
05/19 09:55
阅读数 6.5K

引言

前一篇博文,已经完整的介绍了数据库脱敏的场景及方案,来自京东数科的Sharding-JDBC开源项目通过对数据源中间代理的方式透明化的实现了这个功能,但是,功能虽然实现了,sql兼容的小问题还是很多,比如目前不支持子查询,数据库定义的关键字不允许使用,等等问题,反观我们需要加解密的字段,其实相比业务的sql来说占比非常小,即使遇到了和组件不兼容的地方也可以稍加改动解决掉,所以最后博主给出了一个比较完善的组件集成方案:多数据源模式,需要加解密的数据源和业务其他数据源隔离。即解决了数据库字段加解密的问题,同时也解决了组件对sql的兼容问题。下面是具体的集成步骤以及需要注意的点

引入依赖

            <dependency>
                <groupId>org.apache.shardingsphere</groupId>
                <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
                <version>${sharding.jdbc.version}</version>
            </dependency>

这里需要说明下,虽然采用多数据源兼容后,不能使用组件基于spring boot自动装配功能,但是这里还是建议导入sharding-spring-boot-starter包,因为这个包下内置了配置映射的类,在自定义数据源的时候非常有用

添加sharding数据源配置

#数据库源配置
spring.shardingsphere.datasource.name = ds

spring.shardingsphere.datasource.ds.type = com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.ds.driver-class-name = com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds.jdbc-url = jdbc:mysql://xxx?autoReconnect=true&useUnicode=true&characterEncoding=utf-8
spring.shardingsphere.datasource.ds.username = root
spring.shardingsphere.datasource.ds.password = xxx

spring.shardingsphere.encrypt.encryptors.encryptor_aes.type = aes
spring.shardingsphere.encrypt.encryptors.encryptor_aes.props.aes.key.value = 123456
spring.shardingsphere.encrypt.tables.account.columns.password.plainColumn = password
spring.shardingsphere.encrypt.tables.account.columns.password.cipherColumn = password_encrypt
spring.shardingsphere.encrypt.tables.account.columns.password.encryptor = encryptor_aes

spring.shardingsphere.props.sql.show = true
spring.shardingsphere.props.query.with.cipher.column = true

排除自动装配

@SpringBootApplication(exclude = SpringBootConfiguration.class)

由于导入了starter包,所以这里需要手动排除自动装载类,

业务数据源配置

多数据源后,业务本身的数据源也需要手动配置,默认的spring boot jpa自动转载类会判断上线文中是否存在EntityManagerFactory类,如果有就不会初始化了,所以两个数据源都需要手动配置

@Configuration
@EnableConfigurationProperties(JpaProperties.class)
public class DataSourceConfiguration{

    private final JpaProperties jpaProperties;
    private final Environment environment;

    public DataSourceConfiguration(JpaProperties jpaProperties, Environment environment) {
        this.jpaProperties = jpaProperties;
        this.environment = environment;
    }

    @Primary
    @Bean
    public DataSource dataSource(){
        String prefix = "spring.shardingsphere.datasource.";
        String each = getDataSourceNames(prefix).get(0);
        try {
            return  getDataSource(prefix, each);
        } catch (final ReflectiveOperationException ex) {
            throw new ShardingSphereException("Can't find datasource type!", ex);
        }
    }

    @Primary
    @Bean
    public EntityManagerFactory entityManagerFactory() {
        HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        vendorAdapter.setDatabase(Database.MYSQL);
        vendorAdapter.setGenerateDdl(true);
        vendorAdapter.setShowSql(true);
        LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
        factory.setJpaVendorAdapter(vendorAdapter);
        factory.setPersistenceUnitName("default");
        factory.setPackagesToScan(Constants.BASE_PACKAGES);
        factory.setDataSource(dataSource());
        factory.setJpaPropertyMap(jpaProperties.getProperties());
        factory.afterPropertiesSet();
        return factory.getObject();
    }

    @Bean
    @Primary
    public EntityManager entityManager(EntityManagerFactory entityManagerFactory){
        return SharedEntityManagerCreator.createSharedEntityManager(entityManagerFactory);
    }

    @Primary
    @Bean
    public PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory){
        JpaTransactionManager txManager = new JpaTransactionManager();
        txManager.setEntityManagerFactory(entityManagerFactory);
        return txManager;
    }

    private ListgetDataSourceNames(final String prefix) {
        StandardEnvironment standardEnv = (StandardEnvironment) environment;
        standardEnv.setIgnoreUnresolvableNestedPlaceholders(true);
        return null == standardEnv.getProperty(prefix + "name")
                ? new InlineExpressionParser(standardEnv.getProperty(prefix + "names")).splitAndEvaluate() : Collections.singletonList(standardEnv.getProperty(prefix + "name"));
    }

    @SuppressWarnings("unchecked")
    private DataSource getDataSource(final String prefix, final String dataSourceName) throws ReflectiveOperationException {
        Map dataSourceProps = PropertyUtil.handle(environment, prefix + dataSourceName.trim(), Map.class);
        Preconditions.checkState(!dataSourceProps.isEmpty(), "Wrong datasource properties!");
        DataSource result = DataSourceUtil.getDataSource(dataSourceProps.get("type").toString(), dataSourceProps);
        DataSourcePropertiesSetterHolder.getDataSourcePropertiesSetterByType(dataSourceProps.get("type").toString()).ifPresent(
                dataSourcePropertiesSetter -> dataSourcePropertiesSetter.propertiesSet(environment, prefix, dataSourceName, result));
        return result;
    }
}

上面代码需要注意三个地方,一是数据源的配置,是以sharding的配置来解析获得的,是因为我们已经集成过了,不想改动配置,所以如此,如果还没集成过,可以直接使用spring 配置数据源的方式配置即可。二是EntityManager的初始化,通过SharedEntityManagerCreator包装了下,是因为我们业务的查询通过继承SimpleJpaRepository来扩展功能的,通过SharedEntityManagerCreator包装保留了SimpleJpaRepository内部定义的事务功能。三是需要给所有的业务数据源的配置添加 @Primary注解,让sprign上下文默认使用业务数据源

加解密数据源配置

/**
 * @author: kl @kailing.pub
 * @date: 2020/5/18
 */
@Configuration
@EnableConfigurationProperties({JpaProperties.class,SpringBootEncryptRuleConfigurationProperties.class, SpringBootPropertiesConfigurationProperties.class})
@AutoConfigureAfter({DataSourceAutoConfiguration.class, DataSourceConfiguration.class})
public class EncryptDataSourceConfiguration {

    private final SpringBootPropertiesConfigurationProperties props;
    private final SpringBootEncryptRuleConfigurationProperties encryptRule;
    private final JpaProperties jpaProperties;
    private final DataSource dataSource;

    public EncryptDataSourceConfiguration(SpringBootPropertiesConfigurationProperties props, SpringBootEncryptRuleConfigurationProperties encryptRule, JpaProperties jpaProperties, DataSource dataSource) {
        this.props = props;
        this.encryptRule = encryptRule;
        this.jpaProperties = jpaProperties;
        this.dataSource = dataSource;
    }

    @Bean
    public DataSource encryptDataSource() throws SQLException {
        return EncryptDataSourceFactory.createDataSource(dataSource, new EncryptRuleConfigurationYamlSwapper().swap(encryptRule), props.getProps());
    }

    @Bean
    public EntityManagerFactory encryptEntityManagerFactory() throws SQLException {
        HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
        vendorAdapter.setDatabase(Database.MYSQL);
        LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
        factory.setJpaVendorAdapter(vendorAdapter);
        factory.setPersistenceUnitName("encryptPersistenceUnit");
        factory.setPackagesToScan(Constants.BASE_PACKAGES);
        factory.setDataSource(encryptDataSource());
        factory.setJpaPropertyMap(jpaProperties.getProperties());
        factory.afterPropertiesSet();
        return factory.getObject();
    }

    @Bean
    public EntityManager encryptEntityManager() throws SQLException {
        return SharedEntityManagerCreator.createSharedEntityManager(encryptEntityManagerFactory());
    }

    @Bean
    public PlatformTransactionManager encryptTransactionManager() throws SQLException {
        JpaTransactionManager txManager = new JpaTransactionManager();
        txManager.setEntityManagerFactory(encryptEntityManagerFactory());
        return txManager;
    }
}

加解密数据源的源来自于业务数据源,只是在这里给业务数据源又代理了一层加解密的逻辑。加解密的规则配置采用了sharding-spring-boot-starter包中的映射类,所以可以保留和spring boot配置方式一致。

加解密数据源的使用

在使用时,因为默认使用的是业务数据源,所以需要在需要加解密的地方通过@Qualifier("encryptEntityManager")显示的注入加解密的数据源代理,如:

@Repository
public class AccountRepository extends AbstractJpaRepository {

    public AccountRepository(@Qualifier("encryptEntityManager") EntityManager em) {
        super(AccountModel.class, em);
    }

    @Override
    @Transactional(transactionManager = "encryptTransactionManager")
    public  S save(S entity) {
        return super.save(entity);
    }
}

另,需要手动指定加解密数据源的事务管理器。

如果你使用了JpaRepository等接口,在启用@EnableJpaRepositories的注解里,默认装载的业务数据源配置的EntityManagerFactory实例,需要加解密时,要设置如下参数:

@EnableJpaRepositories(basePackages = "com.xxx",entityManagerFactoryRef = "encryptEntityManagerFactory",transactionManagerRef = "encryptTransactionManager")

结语

没有十全十美的组件,Sharding-JDBC的数据脱敏方案已经趋向于完美了。由于组件本身的架构设计,确实不好做到100%的兼容。在发现加解密组件不支持子查询时,博主发现实现这个功能很简单,尝试过向官方提交这个功能的pr。经过对组件的进一步了解发现,从全局考虑实现这个功能非常复杂,也就放弃了,熟悉java的DBA有志之士可以去试试。目前这个多数据源模式可以很好的解决了sql兼容问题,如果有更好的集成方案,欢迎在下面留言交流

作者简介:

陈凯玲,2016年5月加入凯京科技。负责基础架构中间件的迭代,救火队队长。独立博客KL博客(http://www.kailing.pub)博主。

展开阅读全文
打赏
2
9 收藏
分享
加载中
👍
05/22 15:16
回复
举报
更多评论
打赏
1 评论
9 收藏
2
分享
返回顶部
顶部