Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

 

Data Warehouse HIVE Practice

@ConfigurationProperties(prefix = "hive")
@Data
public class HiveDruidConfig {
 
    private String url;
    private String user;
    private String password;
    private String driverClassName;
    private int initialSize;
    private int minIdle;
    private int maxActive;
    private int maxWait;
    private int timeBetweenEvictionRunsMillis;
    private int minEvictableIdleTimeMillis;
    private String validationQuery;
    private boolean testWhileIdle;
    private boolean testOnBorrow;
    private boolean testOnReturn;
    private boolean poolPreparedStatements;
    private int maxPoolPreparedStatementPerConnectionSize;
 
    @Bean(name = "hiveDruidDataSource")
    @Qualifier("hiveDruidDataSource")
    public DataSource dataSource() {
        DruidDataSource datasource = new DruidDataSource();
        datasource.setUrl(url);
        datasource.setUsername(user);
        datasource.setPassword(password);
        datasource.setDriverClassName(driverClassName);
 
        // pool configuration
        datasource.setInitialSize(initialSize);
        datasource.setMinIdle(minIdle);
        datasource.setMaxActive(maxActive);
        datasource.setMaxWait(maxWait);
        datasource.setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
        datasource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
        datasource.setValidationQuery(validationQuery);
        datasource.setTestWhileIdle(testWhileIdle);
        datasource.setTestOnBorrow(testOnBorrow);
        datasource.setTestOnReturn(testOnReturn);
        datasource.setPoolPreparedStatements(poolPreparedStatements);
        datasource.setMaxPoolPreparedStatementPerConnectionSize(maxPoolPreparedStatementPerConnectionSize);
        return datasource;
    }
 
    @Bean(name = "hiveDruidTemplate")
    public JdbcTemplate hiveDruidTemplate(@Qualifier("hiveDruidDataSource") DataSource dataSource) {
        return new JdbcTemplate(dataSource);
    }

This code is an example of configuring a Hive database connection using Spring Boot and Druid connection pool. It shows how to@ConfigurationPropertiesAnnotations to simplify the binding of configuration properties and how to use Spring@BeanAnnotations to create and manageDataSourceandJdbcTemplateBean. The following is a detailed explanation of this code:

1. @ConfigurationProperties(prefix = "hive")

• effect: This annotation is used to add configuration files (such asapplication.propertiesorapplication.yml) is prefixed withhiveThe properties are bound to the current class (HiveDruidConfig) fields. This makes it very convenient and centralized to configure database connections and other related parameters.
• example: If you areapplication.propertiesThere arehive.url=jdbc:hive2://...,SourlThe fields will be automatically assigned values.

2. @Data

• effect: This annotation comes from the Lombok library, which automatically generates getter, setter, equals, hashCode, and toString methods for the fields of the class. This reduces boilerplate code and makes the class more concise.

3. Field Definition

• The class defines several private fields, which correspond to the configuration parameters of the Hive database connection and the Druid connection pool.

4. dataSource()method

• effect：This method uses@Beanannotation, indicating that it will return a Bean, which will be managed by the Spring container. In this example, it creates and configures aDruidDataSourceInstance, which is the implementation of Druid connection pool.
• Configuration: The various properties of DruidDataSource are set by setter methods. These properties come from the values ​​of class fields, which are set by@ConfigurationPropertiesObtained from the configuration file.
• return value: The method returns the configuredDruidDataSourceInstance, the Spring container will register it as ahiveDruidDataSourceBeans.

5. hiveDruidTemplate()method

• effect: This method also uses@BeanNote that it creates aJdbcTemplateInstance, this instance is used to simplify database operations.
• parameter: The method receives aDataSourceType parameters, through@Qualifier("hiveDruidDataSource")The annotation specifies the use ofhiveDruidDataSourceDataSource Bean.
• return value: The method returns the configuredJdbcTemplateInstance, the Spring container will register it as ahiveDruidTemplateBeans.

Summarize

This code shows how to use@ConfigurationPropertiesand@BeanAnnotations are used to configure and manage database connections. By using the Druid connection pool, the performance and stability of database operations can be further improved.JdbcTemplate, developers can perform database operations more conveniently without writing a lot of JDBC code.