Batch with Salesforce

The task

I was given a task that seemed to be very simple. We had some logic that would collect accounts from the CRM so that we could then send Emails and SMS to contacts on the account.

Since all the work seemed to be CRM only we decided to write all the code in the salesforce apex classes and run a scheduled job.

For the SMS we used twillio, I must say they have a very nice addon to salesforce and it works very nicely.

Salesforce has added a new feature for a single contact to be associated with multiple accounts (see shared_contacts). Since we use this feature, when I send an email or sms I need to select the contacts attached to the account, and the associated contracts, then based on a flag of ours decide if we send them the Email and SMS.

One issue we did have with the SMS is that we would send two SMS to each contact and the order was important (twilio does not promise order). So we had to add a 5sec wait between each SMS call (Salesforce does not support sleep so I added a utility ApexUtil).

Production

I wrote the code and even tested it, and all was fine and worked great. Then came all the problems when we tried to run it in production. I was surprised that I ran into problems even with only a few accounts (about 10).

It seems that salesforce has a lot of limitations on what you can do from apex classes.

Select limit

The first limitation that I hit was a 100 selects per run. At first I did not know how I hit this limit. After some code review, I saw some of the issues.

Email Templates

Depending on the account and parameters we have different Email templates. So for each account I was doing a select for the template. First fix was to load all templates into a map with the name as the key. Since I don’t have that many templates I could load them all, and it would cost me only one select.

Contacts hell

My next issue what that I first selected all the contacts associated with the account, and if there were none then I loaded the contacts from the association (the new feature mentioned above). This of course brought with it up to 2 selects per account.

The solution for this was to add the main contacts to be part of the original select, so my statement now looks like:

SELECT Id, Name, … , (SELECT Id, Name, MobilePhone, …  FROM Contacts where ...) 
FROM Account WHERE ...

This will return within the account object a contacts object that can be used in the code. Then only if there are no contacts will I select the associated contacts (in my case this was the rare case). So for example I added a method to get the contacts from the account:

public List<Contact getContacts(Account account){
    if (account.Contacts.size()0)
        return account.Contacts;
    return [SELECT Id, Name, MobilePhone, ...
    FROM Contact
    WHERE ... AND Id IN (SELECT ContactId FROM AccountContactRelation WHERE AccountId = :account.Id)];
}

So I finally solved my select issues. The next issue that I hit was that nice sleep for about 5sec for each SMS - the process timed out. There seems to be limitations on this too.

So I decided to change my direction and to use Salesforce batch. At first it looks very simple, and the skeleton of the code looks like:

global class PaymentNotificationBatch implements Database.Batchable<SObject>, 
Database.stateful, Database.AllowsCallouts {
  global final String query;
  public Integer total = 0;
  global Database.QueryLocator start(Database.BatchableContext BC){
     return Database.getQueryLocator(PaymentNotifications.query());
  }

  global void execute(Database.BatchableContext BC,List<Account> scope){
     total += scope.size();
     PaymentNotifications pn = new PaymentNotifications();
     for(Account account : scope){
        System.debug('account: ' + account.Name);
        pn.processAccount(account);
     }
     pn.processSMSBatch();
  }

  global voidfinish*Database.BatchableContext BC){
     System.debug('PaymentNotificationBatch finish: ' +  total);
  }
}

Futures

In my original code, an external call from an apex (to send SMS) must be done in a future call. I am not sure for the reason, but I think it is so that the transaction does not get stuck on an external call.

All batch processes are done in a future call, and a future cannot call a future (another limitation).

Since some of my code is generic and it might be called from batch and maybe from apex, I needed to support both. Lucky Salesforce does have a method to check if I am in batch mode or not.

Queueable

Well as you might guess this did not fix my problems just moved the on to another place. Since I am calling twilio as part of my process, I started to get exceptions that you cannot call external systems when the transaction has not finished, and that the solution for this is futures (though in batch you cannot use futures :( ). So I understood that I needed to move all the SMS requests out of the main code and to run it without a transaction.

I found that there is another solution to use in batch processing that should solve the future issue Queueable (it is not enabled out of the box and you need to add the interface Database. AllowsCallouts to your batch class). So I set it so that each SMS will be sent in a queue that is not part of the transaction. Well guess what, that’s right you can only use one queueable in a batch process (how many more limitations are there??).

So back to the drawing board, I created a hash map to store all the SMS’s that I needed to send. Then in the main batch application I can call all the SMS requests but send them on a queueable so that they don’t run under the main thread.

Conclusion

Salesforce CRM has a lot of features and is a good platform. On the other hand it has a lot of limitations that you don’t think about before you dive in.

So before you start a project on salesforce - do your homework first and check what salesforce can or can’t do.