Apex anonymous block will error out because when object has more than 200,000+ records, select clause has to be to efficient. E.g. in above query deleteFlag__c is not indexed and more than half of the records have deleteFlag__c.

Error Message : Non-selective query against large object type (more than 200000 rows). Consider an indexed filter or contact salesforce.com about custom indexing. Even if a field is indexed a filter might still not be selective when: 1. The filter value includes null (for instance binding with a list that contains null) 2. Data skew exists whereby the number of matching rows is very large (for instance, filtering for a particular foreign key value that occurs many times)

Possible Fix

As suggested, Use indexed column (e.g. lookup field - foreign key to constraint the return results) as suggested in error.

Wednesday, September 7, 2016

Problem statement
Every time there is change (create/update/delete) on salesforce data, we had to do quite a bit of processing and then update the external system over WS or HTTP callout. When the record is updated, we might be in normal trigger, visual force controller, scheduled job, batch, queue, or future. Sometime we can not update the external system (e.g. trigger has update waiting or in scheduled job), and sometime we should not, as it will delay the current operation and user experience. It is preferable to have it done in asynchronous manner. We also have to support huge batches of changes. E.g. 5000 accounts can be updated, and we will need to do processing and update external system based on that.

Current Salesforce Solution Limitation

Batch Processing
Let's say we use batch processing and combination of trigger. Upon update of records, we start batch processing.

We can only initiate 100 batches.

If change occurs in future method, then we can not call batch directly

Similar concerns for majority of approaches. Hence, we had to use below approach to do the processing.

Approach

We created EventQueue table

When there is change in record, we push the record to EventQueue table

Wrote batch processing to process the data from Event Queue table

At the end of batch process (finish) method, restart the batch if there is still data in Event Queue

Approach Add On

We also wanted to start the batch when we insert records in EventQueue table, hence we don't have to start or stop the batch manually.

Code initiation can be either from trigger or someone can call our API directly with List of record ids

If data is more than 1

Insert all records in Event Queue (1 row per record)

Start Batch Processing

Else if current context in Queue ( System.isQueue() == true )

Insert record in Event Queue

Start Batch Processing

Else if current context in Queue ( System.isScheduled() == true )

Insert record in Event Queue

Start Batch Processing

Else if we are in Batch

If it is our Batch (Even Queue Processing Batch)

Run the main code to do processing and call external system

Else

Insert record in Event Queue

Start Batch Processing

Else if we are in Future

Insert record in Event Queue

Start Batch Processing

Else

Call the main code in via Future to do processing

Here we can ensure, Main code is running either in a separate Future call (one at a time), or in Batch (one at a time). The caller is never blocked because of this processing.

Batch Processing

start()

Queries the EventQueue table for all record

execute()

Call the entry point via API (mentioned as Direct call in above figure)

finish()

If EventQueue still has records (more records got inserted while we were processing), then initial another job.

Approach Add On - Implementation

We also wanted to start the batch when we insert records in EventQueue table, hence we don't have to start or stop the batch manually.

This turned out to be quite complex. As you can start batch processing as you like.

E.g. if we are future context, we can add records in EventQueue table, but can not start Batch processing - Salesforce limitation.

Similar if there is custom batch which is calling our API with let's 20 records, we put in EventQueue, but can not start batch.

Tuesday, August 16, 2016

An internal server error has occurred
An error has occurred while processing your request. The salesforce.com support team has been notified of the problem. If you believe you have additional information that may be of help in reproducing or correcting the error, please contact Salesforce Support. Please indicate the URL of the page you were requesting, any error id shown on this page as well as any other related information. We apologize for the inconvenience.
Thank you again for your patience and assistance. And thanks for using salesforce.com!
Error ID: 368384005-116910 (1415744368)

If you have namespace prefix for your org, the SOQL issued from APEX already have the prefix assigned e.g. below works just fine and doesn't throw any error