3

Spring Integration Example : file inbound-channel-adapter complex spring integration example

I’ve been using Spring Integration for a long time and would like to share some of my implementations that I used in real projects. You will find many complex spring integration examples in this and upcoming posts.

Spring Integration Example

First of all, if you are new to Spring Integration, here is links and official documentation:

Ok, let’s start rolling! Here is the business requirement (user story) for the spring integration example:

  • We will receive 5 different trade files within 15 minutes time frame.
  • Process should wait for all 5 files and process them together.
  • Process will wait 5 files for a given time period. If cant receive all files within the given period, will raise an exception and move files to Error folder.
  • Each file should be processed only once.
  • Process should check filename and only process the files with correct filename prefix.

That’s it.  Let’s see this spring integration example. Also please note that some code and configurations are simplified for easy reading.

Spring integration example

Spring integration configuration xml:

<file:inbound-channel-adapter 
            id="channelAdapter"
            channel="fileAggregatorChannel"
            directory="C:/File/Inbound" 
            comparator="lastModifiedComparator"
            filter="compositeFileFilter"
            auto-startup="false"
            scanner="dirScanner">
   <poller default="false" max-messages-per-poll="5" >
       <interval-trigger interval="30000"></interval-trigger>
   </poller>
</file:inbound-channel-adapter>

<beans:bean id="dirScanner" class="org.springframework.integration.file.RecursiveLeafOnlyDirectoryScanner"/>

<beans:bean id="compositeFileFilter" class="com.filter.CompositeFileListFilterForNumberOfFiles">
     <beans:constructor-arg >
         <beans:set>
        	    <beans:ref bean="simplePatternFileListFilter" />
        	    <beans:ref bean="lastModifiedFileFilter" />
        	    <beans:ref bean="acceptOnceFileListFilter" />
        	</beans:set>
     </beans:constructor-arg>

     <beans:property name="waitNumberOfFiles" value="5" />
     <beans:property name="maxWait" value="60" />
     <beans:property name="errorFolder" value="C:/File/Error" />
</beans:bean>	

<beans:bean id="simplePatternFileListFilter" class="org.springframework.integration.file.filters.SimplePatternFileListFilter">
      <beans:constructor-arg value="Trade_*"/>
</beans:bean>

<beans:bean id="lastModifiedFileFilter" class="com.filter.FileFilterOnLastModifiedTime">
      <beans:property name="timeDifference" value="10000"/>
</beans:bean>

<beans:bean id="acceptOnceFileListFilter" class="org.springframework.integration.file.filters.AcceptOnceFileListFilter"/>

Java classes:

package com.filter;

import org.springframework.integration.file.filters.AbstractFileListFilter;

/**
 * Make sure that the file adapter is polling of written completed on the box (might be sftp). 
 * It checks for lastModified time of file. 
 */
public class FileFilterOnLastModifiedTime extends AbstractFileListFilter{

    Long timeDifference = 1000L;

    /**
     * Filter a file based on last modification time.
     */
    @Override
    public boolean accept(File file) {

	long lastModified = file.lastModified();
	long currentTime = System.currentTimeMillis();

	return (currentTime - lastModified) > timeDifference ;
    }

    public void setTimeDifference(Long timeDifference) {
         this.timeDifference = timeDifference;
    }

}

 

package com.filter;

import org.springframework.integration.file.filters.FileListFilter;

public class CompositeFileListFilterForNumberOfFiles implements FileListFilter{

    private int waitNumberOfFiles;
    private int maxWait;
    private LocalDateTime firstFileDate = null;
    private String errorFolder;

    public CompositeFileListFilterForNumberOfFiles(Collection> fileFilters) {
          super(fileFilters);
    }

    @Override
    public List filterFiles(F[] files) {

        if (waitNumberOfFiles > files.length) {

            if (firstFileDate == null && files.length >= 1) {
                firstFileDate = LocalDateTime.fromDateFields(new Date());
                logger.info("First file has received at "+firstFileDate.toString());
            }

            if (firstFileDate != null) {

                LocalDateTime now = LocalDateTime.fromDateFields(new Date());
                LocalDateTime expiryDate = firstFileDate.plus(Minutes.minutes(maxWait));

                if (now.isAfter(expiryDate)) {
                    logger.error("First file arrived at "+firstFileDate.toString()+" and waited for "+maxWait+" minutes. Terminating the process!!!");

                    for (F f : files) {
                        File file = (File) f;
                        try {
                            FileUtils.moveFile(file, new File(errorFolder, file.getName()+"_"+now.getMillisOfDay()));
                            logger.info("File moved to error folder. File:"+file.getName());
                        } catch (IOException e) {
                            logger.error("File could not moved to error folder! File:"+file.getName());
                        }
                    }
                    firstFileDate = null;
                }
            }

            if(files.length > 0) {
                logger.warn("Waiting "+waitNumberOfFiles+" number of files but only found "+files.length);
            }
            return new ArrayList();
        }

        List results = new ArrayList(Arrays.asList(files));
        for (FileListFilter fileFilter : this.fileFilters) {

            if(results.size() == 0) {
		break;
	   }
	   List currentResults = fileFilter.filterFiles((F[])results.toArray());
	   results.retainAll(currentResults);
        }

        firstFileDate = null;
        return results;
    }
}

As a reminder, You will find many more complex spring integration examples in the upcoming posts. Follow us for more spring integration examples!

Filed in: Java, Software Development Tags: , , , , ,

Get Updates

Related Posts

3 Responses to "Spring Integration Example : file inbound-channel-adapter complex spring integration example"

  1. Rob Legier says:

    Hello. Excellent job. I did not expect this. This is a outstanding articles. Thanks!

  2. Jonathan Rovers says:

    This post was very nicely written, and it also contains many useful facts. I enjoyed your distinguished way of writing this post. Thanks, you have made it easy for me to understand.

Leave a Reply

Submit Comment

© Deha Solutions. All rights reserved.