Wednesday, 28 September 2011

Concurrent Request chaining.

This is not going to be another definition of what multi-threading or concurrent process execution is, but to findout if you've been another culprit of concurrent request chaining. So what is Concurrent Request Chaining? a simple and perhaps most basic defintion is HTTP GET/POST. The protocol provides a mechanism to accept and process multiple requests via the same session using chaining. As you would appreciate, there are several techniques and built-in objects and types to employ in a multi-threading execution enviroment, to allow a single thread at a time into a particular section of request block thus allowing protection to example variables(make it thread safe) or data from being corrupted by simultaneous modifications from different request coming from the same session.

Although these multi-threaded objects and types provides the answer to the scenario described above, they are not the answer to all business rules. And hence do require proper usage to attain the correct locking for specific instances.

Let's take this business rule for example: A deposit function should be synchronized, if calls to the function are coming from the same account number. This rule is similar to the underlying HTTP request chain explained above.

Noticed there are several approaches to this rule, but how do you provide a robust solution to this, using the concurrent objects and types provided by your JDK? I suppose you've started to think through dozens of possible canditate solutions already. Does each approach work well for the scenario, is the approach lightweight enough?

A classic solution to this will be to join the threads, which if not done properly open doors for dreadful resource issues. Another solution to this will be to use the java.util.concurrent.locks.ReentrantLock and java.util.HashMap to hold a queue of objects to process. This approach gives you the flexibility to lock each process for a specific period before reentrant.
  private Map<Long, ReentrantLock> queue= new HashMap<Long, ReentrantLock>();
  public void AccountDeposit(Long accountNumber)
  {
    addWork(accountNumber);

    ReentrantLock lock = queue.get(accountNumber);
    try
    {
      if (lock.tryLock(1, TimeUnit.MINUTES))
      {
        // Perform Deposit Routine
      }
    }
    catch (InterruptedException e)
    {
      //do some clean up
    }
    finally
    {
      lock.unlock();
    }
  }

  private void addWork(Long accountNumber)
  {
    if (!queue.keySet().contains(accountNumber))
    {
      queue.put(accountNumber, new ReentrantLock());
    }
  }
But if you look closely, the above technique is avoiding adding same Account Number to the queue. If I were you, I will correct this so the queue can hold duplicate customer account numbers to avoid potential deposit calls from been ignored. Here again I will ask you to take very close look, is this approach robust enough? will this be able to handle various data source requests without any potential resource issues in the near future?

Another solution is to manage the pool of threads. This gives you a more flexible way of asking a thread to hold a queue of specific Account Number. Put the deposit and its member objects in a separate instance worker thread, this makes the member objects local to a specific execution unit. Let's take below worker class as an example.
public class DepositHandlerThread extends Thread
{
  public long busyStartTime;
  public static final int MAX_SIZE = 10;
  public final AtomicInteger idUnderProcess = new AtomicInteger();
  private final BlockingQueue<DepositToken> queue = new ArrayBlockingQueue<DepositToken>(MAX_SIZE);

  public static class DepositToken
  {
    private int clientId;

    public DepositToken(int clientId)
    {
      this.clientId = clientId;
    }
  }

  public DepositHandlerThread()
  {
  }

  public void run()
  {
    DepositToken token;
    try
    {
      while ((token =queue.take()) != null)
      {
        // perform the deposit routine
        idUnderProcess.set(token.clientId);
        doDeposit(token);

        if (!queue.isEmpty())
        {
          busyStartTime = 0;
        }
      }
    }
    catch (InterruptedException e)
    {}

  }

  public void doDeposit(DepositToken tokekn)
  {
    /** Some routine here to perform deposit with Token holding client id **/
  }

  public void addWork(DepositToken token)
  {
    try
    {
      busyStartTime = System.currentTimeMillis();
      queue.put(token);
    }
    catch (InterruptedException e)
    {
      throw new AssertionError(e);
    }
  }
}
It accepts and process requests as and when they arrive. Within the controller class which accepts/fetch and process the Deposit Request, you can then manage a pool of worker threads by assigning tasks to avoid same Account Number request executed by seperate threads. This ensures that, a worker currently dealing with a subjected Account Number, has the pile which does not overlaps the execution process, using thread-safe non-blocking FIFO queue. See below example:

Controller class:

  class DepositHandler extends Thread
  {
    int tc;
    private DepositHandlerThread threadPool[];

    public DepositHandler()
    {
      // start pool
      tc = 3;
      threadPool = new DepositHandlerThread[tc];
      for (int i = 0; i < tc; i++)
      {
        threadPool[i] = new DepositHandlerThread();
        threadPool[i].start();
      }
      Thread.yield();
    }

    public void run()
    {
    
    while(/**You have data available from or at some data source**/)
     {
      // I will simulate the situation with a static value
      final Long accountNumber= 123343460;
      boolean taskAssigned = false;

      // Control the pool of threads
      for (int i = 0; i < tc; i++)
      {
        // Check to make sure a live thread is currently working with
        // the ID in hand and if so add the new task to its pile.
        if ((threadPool[i].busyStartTime != 0) && (threadPool[i].isAlive()) && threadPool[i].accountNoUnderProcess .get() == accountNumber)
        {
          // add the new client AccountNumber to queue
          threadPool[i].addWork(new DepositToken(accountNumber));
          taskAssigned = true;
          break;
        }
      }

      if (!taskAssigned)
      {
        int c = 0;
        do
        {
          if (threadPool[c].busyStartTime == 0)
          {
            if (threadPool[c].isAlive())
            {
              // Add works to idle thread
              threadPool[c].addWork(new DepositToken(accountNumber));
              break;
            }
          }
          c++;
        }
        while (c < tc);
      }
     }
    }
  }
The approach exhibited above is transparent, avoid overlaps and future proven, to say the least, it does not force any excessive locking on the process and therefore provide a lightweight thread management process to handle the scenario at hand.

Disclaimer
The code above has not been tested, it has been written to demonstrate request chaining.

Tuesday, 28 June 2011

Capitalise First Letters of String (Useful for Name Conversion)

There are several java utils which can walkthrough words and turn first characters to upper cases - yet if you're like me looking to implement something pretty simple but powerful and also can be easily customized in various ways, then DIY sounds like the way forward.

Below example is something more simple for beginners to start with.

public final class MyUtils
{
  /**
   * Capitalize the First characters of passed Name. If the Name passed are separated by whitespaces, dots, hyphen or apostrophe, the first
   * characters will be capitalized. Example:
   * 

* The name passed = bright dadson.code-nathan o'brian, NameCapitalize(..) will change it to Bright Dadson.Code-Nathan O'Brian *

* @param string * @return */ public static String mameCapitalize(String string) { if (!(string == null)) { char[] replacer = string.toCharArray(); for (int i = 0; i < replacer.length; ++i) { if (((i > -1) && (i == 0)) && Character.isLetter(string.charAt(i))) replacer[i] = Character.toUpperCase(string.charAt(i)); else if ((Character.isWhitespace(string.charAt(i))) || (string.charAt(i) == '-') || (string.charAt(i) == '.' || (string.charAt(i) == '\''))) { if ((i == (replacer.length - 1)) && !Character.isLetter(string.charAt((replacer.length - 1)))) break; if (!(Character.isUpperCase(replacer[(i + 1)]))) { replacer[(i + 1)] = Character.toUpperCase(replacer[(i + 1)]); } } } return String.copyValueOf((replacer)); } return null; } /**To use the util capitalize - something like this**/ public static void main(String... args) { System.out.println(MyUtil.NameCapitalize("bright dadson.code-nathan o'brian")); } }
You are simply creating a pointer in the memory locations to which you modify the address values to upper cases.

Wednesday, 9 June 2010

JavaMail, Commons Mail or Spring Mail - Which does the job?

I thought since the previous culprit focuses on e-mail, it would be essential if I hit the nail on the head with choosing the right API for your e-mail function. For a starter, building e-mail function can be a little fustrating task.

If you are like most developers, you delve in JavaMail API thinking that is it, you have found a boiler plate code which does the job. You then go ahead to customize it to suite your enviroment, finally deployed it to realise your pain has just began.

Take for for instance below JavaMail API generic code:

Email Interface
  public interface EmailEngine
  {
    public void MailTransport(String from, String to, String cc, String subject,
     String message, String doc, String bcc) throws Exception;
  }

Actual Implementation
 public class EmailEngineImpl implements EmailEngine

    public final void MailTransport(String from, String to, String cc, String subject,
     String message, String doc, String bcc)
    {
      Properties props = new Properties();
      props.put(PropsFactory.PropsLookup("mail.host.props"), 
               PropsFactory.PropsLookup("mail.host"));
      Session session = Session.getInstance(props);
      try
      {
        Message msg = new MimeMessage(session);
        MimeMultipart mpRoot = new MimeMultipart();
        MimeMultipart mpAl = new MimeMultipart("alternative");
        MimeBodyPart content = new MimeBodyPart();

        msg.setSubject(subject);
        msg.setSentDate(Today.getDate());
        msg.setFrom(new InternetAddress(from));
        msg.setRecipients(Message.RecipientType.TO, getAddress(to));

        if (cc != null)
        {
          msg.setRecipients(Message.RecipientType.CC, getAddress(cc));
        }
        if (!(bcc == null))
        {
          msg.setRecipients(Message.RecipientType.BCC, getAddress(bcc));
        }
        
        if (doc != null)
        {
          MimeBodyPart p1 = new MimeBodyPart();
          Resource file = new ClassPathResource(doc);
          FileDataSource fds = new FileDataSource(file.getFile())
          {
            public String getContentType()
            {
              return "application/octet-stream";
            }
          };

          p1.setDataHandler(new DataHandler(fds));
          p1.setFileName(fds.getName());
          mpRoot.addBodyPart(p1);
        }
        MimeBodyPart p2 = new MimeBodyPart();
        p2.setText(message);
        mpAl.addBodyPart(p2);

        content.setContent(mpAl);
        mpRoot.addBodyPart(content);

        msg.setContent(mpRoot);
        msg.saveChanges();

        Transport.send(msg);
      }
      catch (Exception ex)
      {
        ex.printStackTrace();
      }
    }
  }
Everything looks fine except you deploy it to realise, Some Email clients typically outlook will work fine but little issues with others. And so you dig around to uncover you'll have to follow the MimeMultiPart structure, WHAP!

You have little time to waste therefore decided to try native Spring Email API. Bravo! Take time to read the API document, did you see the line which says:


Warning regarding multipart mails: Simple MIME messages that just contain HTML text but no inline elements or attachments will work on more or less any email client that is capable of HTML rendering. However, inline elements and attachments are still a major compatibility issue between email clients: It's virtually impossible to get inline elements and attachments working across Microsoft Outlook, Lotus Notes and Mac Mail. Consider choosing a specific multipart mode for your needs: The javadoc on the MULTIPART_MODE constants contains more detailed information.

You avoid the warning and implement Spring Generic Email API as shown below, perhaps tweaking static MULTIPART_MODE to see if it does support various multi email clients. WGL!
NB. Inject org.springframework.mail.javamail.JavaMailSender

  public final void MailTransport(String from, String to, String cc, 
   String subject, String content, String doc, String bcc)
  {
    try
    {
      MimeMessage message = mailSender.createMimeMessage();
      // use the true flag to indicate you need a multipart message
      MimeMessageHelper helper = new MimeMessageHelper(message, true);
      helper.setFrom(from);
      helper.setTo(getAddress(to));
      helper.setSubject(subject.toString());

      if (!(cc == null))
      {
        helper.setCc(getAddress(cc));
      }
      if (!(bcc == null))
      {
        helper.setBcc(getAddress(bcc));
      }

      if (!(doc == null))
      {
        Resource file = new ClassPathResource(doc);
        helper.addAttachment(file.getFilename(), file);
      }

      helper.setText(content);
      mailSender.send(message);
    }
    catch (Exception ex)
    {
      ex.printStackTrace();
    }
  }

If you do have luck with either Generic JavaMail or Spring implementation, I suppose your first reaction will be, why bother trying Apache Commons Email! You want MHO, go ahead and try it. It saves time and does the job. Well each of these API has it flaws, but TMHO Apache Commons Email ticks most boxes. Get the dependency or the jar file onto your classpath and try this;


  public class EmailEngineImpl implements EmailEngine
  {
    public final void MailTransport(String from, String to, String cc, String subject, String message, String doc, String bcc)
    {
      // Create Multi-Part document
      MultiPartEmail email = new MultiPartEmail();
      try
      {
        if (!(doc == null))
        {
          // Create the attachment
          EmailAttachment attachment = new EmailAttachment();
          Resource file = new ClassPathResource(doc);
          
          attachment.setURL(file.getURL());
          attachment.setDisposition(EmailAttachment.ATTACHMENT);
          attachment.setName(file.getFilename());

          // add the attachment
          email.attach(attachment);
        }

        // Set Mail server location
        email.setHostName(PropsFactory.PropsLookup("mail.host"));
        email.setTo(getAddress(to));
        email.setFrom(from);

        if (!(cc == null))
        {
          email.setCc(getAddress(cc));
        }

        if (!(bcc == null))
        {
          email.setBcc(getAddress(bcc));
        }
        email.setSubject(subject);
        email.setMsg(message);

        // Send message
        email.send();
      }
      catch (Exception ex)
      {
        ex.printStackTrace();
      }
    }
  }
I did try these and realised, most little issues like multi client and attachment was dealt with. Like I said this is my opinion let me know what you think. And oh if you wondering WTF is the method getAddress(..), see below:

Implementation for Apache Commons
  private Collection getAddress(String arg)
  {
    Collection address = new ArrayList();
    try
    {
      ArrayList ccArray = new ArrayList();
      StringTokenizer st = new StringTokenizer(arg, ",");
      while (st.hasMoreTokens())
      {
        ccArray.add(new InternetAddress(st.nextToken()));
      }

      for (int i = 0; i < ccArray.size(); i++)
      {
        address.add(ccArray.get(i));
      }

    }
    catch (Exception ex)
    {
      ex.printStackTrace();
    }
    return address;
  }
Implementation for JavaMail or Spring
 private InternetAddress[] getAddress(String arg)
  {
    InternetAddress[] address = null;
    try
    {
      ArrayList ccArray = new ArrayList();
      StringTokenizer st = new StringTokenizer(arg, ",");

      while (st.hasMoreTokens())
      {
        ccArray.add(new InternetAddress(st.nextToken()));
      }

      int size = ccArray.size();
      address = new InternetAddress[size];
      for (int i = 0; i < size; i++)
      {
        address[i] = new InternetAddress(ccArray.get(i).toString());
      }

    }
    catch (Exception ex)
    {
      ex.printStackTrace();
    }
    return address;
  }

Drop me a comment to let me know what you think.

Monday, 7 June 2010

Commons mail, JavaMail or Spring Mail with Maven2 and servlet container dependency issue.

If you are working on Spring Web project with Axis2, Axiom or Apache CXF 2 dependency, unless the application do not have concerns with e-mail confirmation/notification functionality. Else you are likely to run into this culprit.
How do you know you've got this issue? You execute a test application from your IDE to see if an e-mail will be sent, to your surprise everything works perfect. But when you deploy the application to either Tomcat, Jboss or Glassfish server, the message is sent with header and footer details as part of the mesage body. The massage body typically resembles below;
------=_Part_0_25002283.1275298567928
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hello Mark Doe,

This message is found as a culprit.

Regards,

Joe Blahh

------=_Part_0_25002283.1275298567928

Also if you are attaching documents to this email message, the attachment is sent with base64 encode as part of the message body. Typical attachment message will look like this;

------=_Part_0_25002283.1275298567928
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hello Mark Doe,

This message is found as a culprit.

Regards,

Joe Blahh

------=_Part_0_25002283.1275298567928
Content-Type: application/pdf; name="file.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; 
filename="file.pdf"
Content-Description: Some attachment

JVBERi0xLjQNJeLjz9MNCjYzIDAgb2JqDTw8L0xpbmVhcml6ZWQgMS9MIDMxMzE4Mi9PIDY1L0Ug
Mjg2NjY5L04gMS9UIDMxMTgwMi9IIFsgMjgzNiAzNzZdPj4NZW5kb2JqDSAgICAgICAgICAgICAg
DQp4cmVmDQo2MyAxMjcNCjAwMDAwMDAwMTYgMDAwMDAgbg0KMDAwMDAwMzM4MCAwMDAwMCBuDQow
MDAwMDAzNTIzIDAwMDAwIG4NCjAwMDAwMDQzMDcgMDAwMDAgbg0KMDAwMDAwNTEwOSAwMDAwMCBu
DQowMDAwMDA2Mjc5IDAwMDAwIG4NCjAwMDAwMDY0MTAgMDAwMDAgbg0KMDAwMDAwNjU0NiAwMDAw
MCBuDQowMDAwMDA3OTY3IDAwMDAwIG4NCjAwMDAwMDkwMjMgMDAwMDAgbg0KMDAwMDAwOTk0OSAw
MDAwMCBuDQowMDAwMDExMDAwIDAwMDAwIG4NCjAwMDAwMTIwNTkgMDAwMDAgbg0KMDAwMDAxMjky
MCAwMDAwMCBuDQowMDAwMDEyOTU0IDAwMDAwIG4NCjAwMDAwMTI5ODIgMDAwMDAgbg0KMDAwMDAx
.......
CnN0YXJ0eHJlZg0KMTE2DQolJUVPRg0K
------=_Part_0_25002283.1275298567928--

If you've run into this culprit, don't worry too much. Go grab a cuppa coffee, and while you're relaxed look through your Maven dependency to see if you have geronimo-javamail_1.4_spec-1.2 along with geronimo-activation_1.1_spec, because your project build will rely on these. You can do this by issuing below Maven2 commands;
mvn dependency:tree
Optionally, the output parameter can be specified to divert the output to a file:
mvn dependency:tree -Doutput=/path/to/file

Causes:
The described issue is caused by transitive dependencies of Apache CXF 2 or Axiom.

SOLUTION:
To resolve this issue, exclude geronimo-javamail_1.4_spec from the build, and just rely on javax's mail-1.4.x.jar.

    
  org.apache.cxf   
  cxf-rt-frontend-jaxws    
  2.2.6    
            
                  
       org.apache.geronimo.specs            
       geronimo-javamail_1.4_spec        
      
      
      org.apache.geronimo.specs 
      geronimo-activation_1.1_spec
         
 



  
  org.apache.ws.commons.axiom  
  axiom-api  
  1.2.8  
           
      
      org.apache.geronimo.specs 
      geronimo-activation_1.1_spec
     
     
      org.apache.geronimo.specs 
      geronimo-javamail_1.4_spec