Skip to content

How to Develop a Transport Connector Reader

Quick Guide

Short summary over the steps described below:

  1. Prerequisites.
  2. Prepare project structure.
  3. Create class extending ConnectorReadersConfig
  4. Create class extending ConnectReader
  5. Create class implementing ConnectReaderFactory

Connector Readers

Connector Readers handle inbound messages sent from from other systems to IFS Cloud, based on the native protocol (e.g. FTP, SMTP, etc.) used for the communication. IFS Cloud comes with a number of Connector Readers out of the box.

In this document we are going to show you how to develop your own custom Connector Reader. With that said, we recommend you to write a new connector reader only if none of the existing readers serves your requirement.

How to Develop a Connector Reader

Project Preparation

Before you can start with development of your custom Connect Reader you have to prepare a proper ANT project structure.

List vs. Loop

A typical Reader architecture is based on two different steps that can take place on two different cluster nodes in the system. The first step is producing a list of available messages without reading the them, while the second step will process a single message referred by ID obtained by listing.
But for some native protocols it is not possible to create such list, either because it is not possible to list messages without reading them or because it is not possible to retrieve a unique, persistent ID of a message. For this type of protocols IFS Connect framework offers another, one-step approach. The Reader is then looping through all available messages and processing messages one by one. The first approach should be used if possible because of better throughput, performance and scalability. Read more about Connect Reader Architecture.

A complete example project for the List Reader can be found here (the example is located in workspace/fndint/source/fndint/connectors/my_list_connect_reader).
A complete example project for the Loop Reader can be found here (the example is located in workspace/fndint/source/fndint/connectors/my_loop_connect_reader).

So preferably just copy one of the examples to your component structure, adapt to your needs and then open in NetBeans.

Configuration

Connect Reader configuration is accessed through the Setup IFS Connect feature in Solution Manager. To be possible to create a node in Setup IFS Connect corresponding to your reader you have to create as described in Connect Reader and Sender Configuration.

Java Source Code

You have to create at least three Java classes based on abstract classes and interfaces located in the Connect Framework JAR file, ifs-fnd-connect.jar. One of those three classes will be responsible for the configuration mentioned above, the two other are the actual reader implementation and a factory class.

  • Class extending ConnectorReadersConfig.
  • Class extending ConnectReader.
  • Class implementing ConnectReaderFactory.

ConnectorReadersConfig

Start with implementing a class that extends ifs.fnd.connect.config.ConnectorReadersConfig (that in turn extends ifs.fnd.connect.config.Config). This class corresponds to reader configuration in Setup IFS Connect. The class exposes all necessary parameters as public final variables. For that it has to have a static inner class Builder extending ConnectorReadersConfig.Builder, which is responsible for reading of parameters, and a private constructor taking instance of this Builder class as argument. First in the constructor call super(builder). Next copy all the variables you want to have access to from your Reader code from the Builder instance to public final variables in your Config class:

package ifs.fndint.connectreader;  

import ifs.fnd.base.IfsException;  
import ifs.application.fndconnectframework.ConfigInstanceParam;  
import ifs.fnd.connect.config.Config;  
import ifs.fnd.connect.config.ConnectorReadersConfig;  

public class MyListConnectReaderConfig extends ConnectorReadersConfig {  

   // inner Builder class  
   static class Builder extends ConnectorReadersConfig.Builder {  
      private String myVariable;  
      // other variable declarations here ...  

      // method implementations ...  
   }  

   // Config instance  

   public final String myVariable;  
   // other public final variables here ...  

   private MyListConnectReaderConfig(Builder builder) {  
      super(builder);  
      myVariable = builder.myVariable;  
      // copy other variables here ...  
   }  
}

Abstract methods that need to be overridden in the inner Builder class:

protected void init(String name, ConfigInstanceParam param) throws IfsExceptionImplement a switch statement reading all your configuration parameters (with exception of FACTORY_CLASS, EXECUTION_MODE, CREATE_RESPONSE, LOG_LEVEL, MAX_RETRIES, MESSAGE_SELECTOR, DEFAULT_ENCODING, WORK_TIMEOUT and ENABLED that are handled by the framework).
You can use methods like replacePlaceholders() and get*Value() implemented in the super-classes. Store the read values in private member variables:

@Override  
protected void init(String name, ConfigInstanceParam param) throws IfsException {  
   switch (name) {  
      // read all reader specific parameters; common parameters: FACTORY_CLASS,  
      // EXECUTION_MODE, CREATE_RESPONSE, LOG_LEVEL, MAX_RETRIES, MESSAGE_SELECTOR,  
      // DEFAULT_ENCODING, WORK_TIMEOUT and ENABLED are already handled by  
      // the framework and don't need to be read here  
      case "MY_FIRST_PARAMETER":  
         myVariable = replacePlaceholders(getTextValue(name,param));  
         // 'location' parameter defined in the super class is used during routing.  
         // each reader decides meaning of this parameter.  
         location = myVariable;  
         break;  
      case "MY_SECOND_PARAMETER":  
         mySecondVariable = replacePlaceholders(getTextValue(name,param));  

         break;  
      // read other parameters here ...  
   }  
}

protected void postInit() throws IfsException
If necessary, you can validate consistence of you parameters here. You can also create new variables based on combinations of the read ones.
Note that an exception thrown from this method will prevent the server from start, so if the user can enter an invalid combination of parameters it is better to correct the values to a valid combination and write a warning to the log file rather then throwing an exception:

@Override  
protected void postInit() throws IfsException {  
   // any code that has to be executed after all parameters have been read,  
   // e.g. parameter validation, consistency, variables that depend on several parameters, etc...  
   // for example in Mail Reader the 'location' parameter is defined here:  
   location = username+"@"+host+":"+port;  
}

protected Config newConfig()
This method is supposed to return a new instance of your Config (i.e. ConnectorReadersConfig) class:

@Override  
protected Config newConfig() {  
   return new MyListConnectReaderConfig(this);  
}

ConnectReader

Now its a time to implement the Reader itself. You will do it by implementing a parameterized class extending ifs.fnd.connect.readers.ConnectReader<C> (if your Reader is supposed to be based on the List architecture, which is preferred) or ifs.fnd.connect.readers.NolistConnectReader<C> (for Loop Reader), where C is the name of your first class, i.e. the one extending ConnectorReadersConfig.

- List

Your Reader class has to extend ConnectReader:

package ifs.fndint.connectreader;  

import ifs.fnd.connect.readers.ConnectReader;  
import ifs.fnd.util.IoUtil;  


import java.io.File;  
import java.io.IOException;  
import java.nio.file.Files;  
import java.nio.file.Path;  
import java.nio.file.Paths;  
import java.util.*;  


public class MyListConnectReader extends ConnectReader<MyListConnectReaderConfig> {  

   private transient String myMessageId; // instance variable that uniquely identifies a message  

   // method implementations ...  
}

And you have to override following abstract methods:

public void nativeInitReader() throws ReaderFailureException
Initialize all necessary resources used by the reader. The reader implements the AutoCloseable interface, so all resources initiated/open here have to be released/closed in the nativeClose() method.

@Override  
public void nativeInitReader() throws ReaderFailureException {  
   try {  
      // initialize resources used by the reader.  
      // it can be directories that have to be created,  
      // connections, user authentication, etc.  
      // ...  
   }  
   catch(SoftException e) {  
      // some exceptions are maybe just temporary ones...  
      throw new TemporaryFailureException(e, "Error while initializing reader...");  
   }  
   catch(HardException e) {  
      // but typically you want to throw permanent failure on initialization error...  
      throw new PermanentFailureException(e, "Error while initializing reader...");  
   }  
}

public List<String> nativeList() throws ReaderFailureException
This method is implementing the first step in List Reader implementation. Here you create a List containing globally unique IDs of available messages. This method will be called about twice in a minute. Note also that reading of messages given their IDs may be done by another instance of this class that can be located on another cluster node, another physical machine.

@Override  
public List<String> nativeList() throws ReaderFailureException {  
   List<String> messageIds = new ArrayList<>();  
   // some preparations...  
   try {  
      // some work here ...  
      MyMessage[] myMessages = ...  
      for(MyMessage msg: myMessages) {  
         // some other work ...  
         String msgId = ...  
         list.add(msgId);  
      }  
      return messageIds;  
   }  
   catch(AnException e) {  
      // error handling here; throw TemporaryFailureException or PermanentFailureException  
   }  
}

public void nativeInitMessage(String messageId) throws ReaderFailureException
If you have any special resources or variables that have to been initialized per message you can do it here. In necessary, resources initiated/open here have to be released/closed in the nativeCloseMessage() method. This method is called just before nativeRead(), so typically, based on the given message ID, you will prepare and save as instance variables the necessary data that is required for reading and deleting the message later on.

@Override  
public void nativeInitMessage(String messageId) throws ReaderFailureException {  
   // initialization of resources used per message basis ...  
   // decode message ID; it can be file name or something else ...  
   myMessageId = messageId;  
}  

public ConnectReader.Message nativeRead() throws ReaderFailureException
Here you read the message. Use the instance variables prepared by nativeInitMessage() to find the actual message. Note that several instances of this class can read messages in parallel over the entire cluster, depending on the Reader's EXECUTION_MODE. The content of the read message is returned as an instance of the inner class ConnectReader.Message, which encapsulates the read bytes.

@Override  
public Message nativeRead() throws ReaderFailureException {  
   try {  
      // read data in the native way using myMessageId...  
      byte[] data = ...  
      Message msg = new Message(myMessageId, name); // you can send ID as name  
      msg.setData(data);  
      return msg;  
   }  
   catch(AnException e) {  
      // error handling here; throw TemporaryFailureException or PermanentFailureException  
   }  
}

public void nativeDelete() throws ReaderFailureException
This method is called after successful reading and processing of the message. Use the instance variables prepared by nativeInitMessage() to find the actual message and delete it.

@Override  
public void nativeDelete() throws ReaderFailureException {  
   try {  
      // delete message using myMessageId...  
   }  
   catch(AnException e) {  
      // error handling here; throw TemporaryFailureException or PermanentFailureException  
   }  
}

public void nativeCloseMessage() throws PermanentFailureException
Here, if necessary, you close/release all resources open/initiated by nativeInitMessage(), i.e. per message. This method is not declared as abstract in the super class, but has an empty implementation, so you don't need to override it if you don't need to perform any special cleaning action per message basis.

public void nativeClose() throws PermanentFailureException
If necessary you can close/release all resources open/initiated by nativeInitReader() here.

@Override  
public void nativeClose() throws PermanentFailureException {  
   // close all resources open by nativeInitReader() here ...  
}

- Loop

package ifs.fndint.connectreader;  

import ifs.fnd.connect.readers.NolistConnectReader;  
import ifs.fnd.util.IoUtil;  

import java.io.File;  
import java.io.IOException;  
import java.nio.file.Files;  
import java.nio.file.Path;  
import java.nio.file.Paths;  
import java.util.*;  


public class MyLoopConnectReader extends NolistConnectReader<MyLoopConnectReaderConfig> {  

   // method implementations ...  
}

If your Reader class is extending NolistConnectReader, you have to override following abstract methods:

protected void nativeInit() throws ReaderFailureException
Initialize all necessary resources used by the reader. The reader implements the AutoCloseable interface, so all resources initiated/open here have to be released/closed in the nativeClose() method.

@Override  
protected void nativeInit() throws ReaderFailureException {  
   try {  
      // initialize resources used by the reader.  
      // it can be directories that have to be created,  
      // connections, user authentication, etc.  
      // ...  
   }  
   catch(SoftException e) {  
      // some exceptions are maybe just temporary ones...  
      throw new TemporaryFailureException(e, "Error while initializing reader...");  
   }  
   catch(HardException e) {  
      // but typically you want to throw permanent failure on initialization error...  
      throw new PermanentFailureException(e, "Error while initializing reader...");  
   }  
}

protected void nativeLoop() throws ReaderFailureException
Here you loop over all available message and read them one by one. After reading a message you create an instance of ConnectReader.Message and send it to a callback function:
   protected final void processMessage(Message msg) throws ReaderFailureException
located in the super class.

@Override  
protected void nativeLoop() throws ReaderFailureException {  
   // some preparations...  
   try {  
      // some work here ...  
      MyMessage[] myMessages = ...  
      for(MyMessage msg: myMessages) {  
         // some other work ...  
         String msgId = ...  
         Message msg = new Message(msgId, name); // you can send ID as name  
         msg.setData(data);  
         super.processMessage(msg);  
         // everything went ok if you're here - you can safely delete the message  
         // delete message...  
      }  
   }  
   catch(AnException e) {  
      // error handling here; throw TemporaryFailureException or PermanentFailureException  
   }  
}

public void nativeClose() throws PermanentFailureException
If necessary you can close/release all resources open/initiated by nativeInit() here.

@Override  
public void nativeClose() throws PermanentFailureException {  
   // close all resources open by nativeInit() here ...  
}

ConnectReaderFactory

Finally you have to implement a class implementing the interface ifs.fnd.connect.senders.ConnectReaderFactory. Fully qualified name of this class you have to put as the default value of the configuration parameter FACTORY_CLASS in your reader configuration. The class needs to implement only two simple methods that create new instances of the classes you have implemented above:

public ConnectorReadersConfig.Builder newConfigBuilder();
public ConnectReader<? extends ConnectorReadersConfig> newConnectReader();

package ifs.fndint.connectreader;  

import ifs.fnd.connect.config.ConnectorReadersConfig;  
import ifs.fnd.connect.readers.ConnectReader;  
import ifs.fnd.connect.readers.ConnectReaderFactory;  

/ * Custom Reader Factory class registered in the configuration.  
 * The class is responsible for construction of both the reader class  
 * itself and the Config class.  
 */  
public class MyListConnectReaderFactory implements ConnectReaderFactory {  

   @Override  
   public ConnectorReadersConfig.Builder newConfigBuilder() {  
      // constructs new instance of Config.Builder  
      return new MyListConnectReaderConfig.Builder();  
   }  

   @Override  
   public ConnectReader<? extends ConnectorReadersConfig> newConnectReader() {  
      // constructs new instance of Reader  
      return new MyListConnectReader();  
   }  
}

Error Handling

The ConnectReader class defines two inner Exception classes sharing a common super class ReaderFailureException. Those are:

`TemporaryFailureException

Throw this exception if the failure is of a temporary kind and there is a chance that the *Reader* will recover from the failure..  



PermanentFailureException
`This exception is supposed to be thrown if it is not possible to recover from the error.