2015年1月15日 星期四

Spring Batch csv to csv 專案

Spring Batch csv to csv 專案

本範例實作說明:
本篇主要使用 Spring Batch,做一個基本測試專案,
    此Batch 為 讀取csv檔 轉換為 csv檔 的功能。
使用到 Spring Batch 的 ItemReader、ItemProcessor和ItemWriter 。

一、範例開發準備工具

作業系統:Windows 7
開發工具:Eclipse Java EE IDE for Web Developers Juno Service Release2
JAVA JDK : JDK1.7.0_60
MS_SQL 2012
其它相關:Maven 4.0
  Springframework 4.1.3.RELEASE
  Springframework batch 3.0.2.RELEASE
     Junit-test 4.11

二、新增Maven範例專案(請參考 Spring Batch Hello World Example 專案)

三、相關resoures

建立resoures 路徑
src/main/reresoures
建立設定檔存放路徑 
src/main/reresoures/config
建立Spring batch 設定檔存放路徑 
src/main/reresoures/job
建立匯入csv測試資料檔存放路徑 
src/test/reresoures/csv

  圖1 建立相關 resoures

四、建立資料庫資訊檔 job-database.xml(請參考 Spring Batch Hello World Example 專案)

五、建立Spring batch 設定檔存放路徑
src/main/reresoures/config/job-context.xml
src/main/reresoures/config/job/spring-batch-context.xml

spring-batch-context.xml內容:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">

    <job id="inMemoryJob" xmlns="http://www.springframework.org/schema/batch">
        <step id="inMemoryStep">
            <tasklet>
                <chunk 
                    reader="itemReader"
                    processor="itemProcessor"
                    writer="itemWriter" 
                    commit-interval="5">
                </chunk>
            </tasklet>
        </step>
    </job>

    <bean id="itemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
        <property name="resource" value="#{jobParameters['input.file']}" />
        <property name="lineMapper">
         <!-- 直接拋回一筆資料 -->
            <bean class="org.springframework.batch.item.file.mapping.PassThroughLineMapper" />
        </property>
        <property name="strict" value="true" />
    </bean>

    <bean id="itemProcessor" class="com.sample.SampleItemProcessor" />
    
    <bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
        <property name="resource" value="#{jobParameters['output.file']}" />
        <!-- 取得一筆資料,回傳 String -->
        <property name="lineAggregator">
         <!-- 取得資料後直接toString()拋出 -->
            <bean class="org.springframework.batch.item.file.transform.PassThroughLineAggregator"/>
        </property>
    </bean>

</beans>


job-context.xml內容:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans 
        http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">
        
    <bean id="jobRepository"
  class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
  <property name="dataSource" ref="dataSource" />
  <property name="transactionManager" ref="transactionManager" />
  <property name="databaseType" value="sqlserver" />
 </bean>
 
 <bean id="jobLauncher"  class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
  <property name="jobRepository" ref="jobRepository" />
 </bean>

</beans>

六、建立匯入csv測試資料檔
資料存放位置: src/test/reresoures/csv/input.csv

1001,"213,12",980,levin , 2013/1/02
1002,"220,20",1080,tom 1 , 2013/2/4
1003,"252,19",2200,tom 2 , 2013/11/10
1003,"248,19",1500,tom 3 , 2013/10/20
1003,"552,18",2800,tom 4 , 2013/6/15
1003,"323,78",1900,tom 5 , 2013/5/12


七、建立讀取資料項目程式 SampleItemProcessor.java
檔案路徑: src/main/java/com/sample/SampleItemProcessor.java

package com.sample;

import org.springframework.batch.item.ItemProcessor;

public class SampleItemProcessor implements ItemProcessor {

    public String process(String item) throws Exception {
        System.out.println(item);
        return item;
    }
}
八、執行job ApplicationTest.java
檔案路徑: src/test/java/com/sample/ApplicationTest.java
package com.sample;


import static org.junit.Assert.assertEquals;

import java.util.HashMap;
import java.util.Map;

import org.junit.Test;
import org.springframework.batch.core.BatchStatus;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameter;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class ApplicationTest {

 @Test
 @SuppressWarnings("resource")
        public void launchJob() throws Exception {
     
     ApplicationContext context = new ClassPathXmlApplicationContext(
                "job/spring-batch-context.xml", "config/job-context.xml",
                "config/job-database.xml");
  
           Job job = (Job) context.getBean("inMemoryJob");
        
            JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
        
            Map jobParametersMap = new HashMap();
            jobParametersMap.put("input.file", new JobParameter("file:src/test/resources/csv/input.csv"));
            jobParametersMap.put("output.file", new JobParameter("file:target/test-outputs/output.csv"));

            JobExecution jobExecution = launcher.run(job, new JobParameters(jobParametersMap));
        
            assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus());
            System.out.println("執行結果 = " + jobExecution.getStatus());
            System.out.println("執行程式結束.");
    }
}


九、執行結果

圖一.

圖二. 產生檔案及內容











其它文章

沒有留言:

張貼留言

標籤

Oracle (150) Oracle DB (144) Oracle_DB (143) Oracle SQL (135) JAVA (84) css-基本類 (65) MySQL (59) CSS Selector (58) jQuery (49) JavaScript-基本類 (39) Spring Boot (38) JavaScript (37) JavaScript HTML DOM (37) JavaScript-HTML_DOM (36) CSS3 (30) JAVA-基本類 (28) jQuery UI (27) Apache (23) Oracle GROUP BY (20) datepicker (20) Android (18) Oracle Date (17) c (17) JAVA-lang套件 (16) Linux (16) Oracle Sub Query (16) Spring-基本類 (16) jQuery-基本類 (16) MySQL-進階系列教學 (15) Android基本類 (14) Grails (14) Oracle join (14) SQLite (13) Spring (13) WIN7-基本類 (13) grails-基本類 (13) linux cent os (13) CKEditor (12) JAVA-流程控制類 (12) JAVA_Spring (12) PHP (11) Spring MVC (11) MySQL-基本系列教學 (10) Notepad (10) Notepad++ (10) SQLite for java (10) Windows (10) c/c++ (10) eclipse (9) jQuery-Selector (9) sqldeveloper (9) DB_Toad (8) JAVA_IDE_Eclipse (8) JavaScript-String類 (8) MySQL DB Toad (8) MySQL-DATE相關 (8) MySQL-函式相關 (8) Spring Bean (8) Android Studio (7) HTML5 (7) Hibernate (7) JAVA-OCWCD (7) JavaScript-陣列類 (7) Docker (6) JAVA-程式分享 (6) JAVA.util套件 (6) JavaScript-數學類 (6) MinGw (6) MySQL-其它類 (6) Servlet (6) centos (6) Apache_Tomcat (5) Apache套件_POI (5) CSS (5) JavaScript-Date物件 (5) JavaScript-其它類 (5) PostgreSQL (5) httpd (5) log4j (5) 基本資訊 (5) 開發工具 (5) CSS Properties (4) Dev-C++ (4) IntelliJ IDEA (4) Oracle DDL (4) Sublime (4) TortoiseSVN (4) apache_Maven (4) Android NDK (3) Eclipse IDE for C/C++ (3) Hibernate-基本類 (3) JAVA-問題 (3) JAVA-綀習分享 (3) JVM (3) Linux 指令 (3) Proxy Server (3) Spring Mobile (3) Spring web (3) Squid (3) VirtualBox (3) maven (3) zk (3) 生活其它 (3) Bootstrap (2) Filter (2) JAVA_IO (2) JAVA_其它_itext套件 (2) JBoss-問題 (2) JSP (2) Jboss (2) Listener (2) MySQL-語法快速查詢 (2) Spring AOP (2) Spring Batch (2) Spring Boot Actuator (2) Spring i18n (2) Subversive (2) Tomcat 8 (2) UML (2) WebJars (2) WinMerge (2) c++ (2) c語言綀習題 (2) jQuery Mobile (2) jQuery-事件處理 (2) jQuery-套件類 (2) putty (2) svn (2) weblogic (2) Apache_JMeter (1) Apache套件_BeanUtils (1) Apache套件_StringUtils (1) Base64 (1) Google API (1) HTML5-基本類 (1) Heap (1) JAVA 7 (1) JAVA SE 、JAVA EE、JAVA ME (1) JAVA 日期 (1) JAVA-OCJP (1) JAVA-WEB (1) JAVA_IDE (1) JAVA其它 (1) JBoss Server (1) JDK (1) JMX (1) JRE (1) Java RMI (1) Java String (1) Joda Time (1) Linux_其它 (1) MySQL教學 (1) Oracle_VirtualBox (1) SQL Server (1) SWT (1) Session (1) Stack (1) Struts 2 (1) Tool (1) ZK Studio (1) csv (1) grails-其它類 (1) jQuery-進階 (1) java mail (1) java web (1) java8 (1) jsoup (1) mockmvc (1) modules (1) tomcat (1) win10 (1) 其它類 (1) 圖片工具 (1) 模擬器 (1) 讀書分享 (1) 開發資訊 (1)

精選文章

初學 Java 的 HelloWorld 程式

撰寫一個JAVA程式 public class HelloWorld{ public static void main(String[ ] args){ System.out.println("我第一支Java程式!!"); } } ...