Batch Processor - FAQ
1.1. Navigate to the Merchant Facility
Click on the Merchant Facility Tab
Select "All Merchant Facilities"
Click into the Primary and Active Merchant Facility
1.2. Start/Stop the Batch Processor
The Batch Payments Processors section is about 1/4 of the way down the screen.
- Press the Button "START" to Start. Note, after starting, the label changes to STOP.
- If already started, press the "STOP" button to stop. Note, after stopping, the label changes to START.
2.1. Navigate to Setup > Scheduled Jobs
- Click on the setup cog (top right)
- Search "Scheduled Jobs" and click into Scheduled Jobs
2.2. Look for duplicate job names beginning with "batchPaymentsProcessor" and delete
- Sort current jobs by Job Name
- Check to see if there are multiple scheduled jobs with the name starting with "batchPaymentsProcessor".
- ONLY IF THERE IS MULTIPLE. Press Del for each processor. Make sure you do the next step of restarting the batch processor.
2.3. Restart Batch Processors
See procedure: How do I start or stop the Batch Processor and START the Batch Payments Processor.
3.1. Phantom Batch Processor Introduction
On rare occasions, a Batch Processor may get stuck in a processing status of Queued. This seems to occur when Salesforce has done upgrades.
Symptoms of this happening are when Payment Txn Status's get stuck in Receipting Complete Status and the Batch Processor status on the Merchant Facility is showing as running.
From version 7.7 onwards, an Error Log is created indicating this condition exists.
Apart from Payment Txn's staying at a status of "Receipting Complete", another indication of Phantom or Duplicate Batch Processors can be seen up setup > Search "Apex Jobs". In this screen, if you observe the "batchProcessor" job NOT showing "Completed" status and ONLY Aborted Status, then that is sign that phantom or duplicate Batch Processor jobs issue exists. NOTE: Aborted Status and "Completed" status showing is NORMAL and all is ok.
The below screenshot shows the issue.
3.2. Check for duplicates
Before continuing with this procedure, first check to see if multiple processors are currently scheduled. If they are, you can simply stop those and all should be ok. If you are still observing issues or getting error reports after that, then continue with this procedure.
See procedure: How do I check for an cancel duplicate Batch Processors running.
3.3. Stop the Batch Processors
See procedure How do I start or stop the Batch Processor on how to STOP the Processors
3.4. Query Phantom Batch Processor Jobs from Developer Console
3.4.1. Open the Developer Console
- Top right click on the settings cog
- Followed by Developer Console
3.4.2. Run query on Batch Processors
- Click on the query tab.
- Paste “SELECT Id,Status,JobType,ApexClass.Name,CreatedDate,LastProcessed,JobItemsProcessed,NumberOfErrors FROM AsyncApexJob WHERE (JobType IN ('BatchApex','ScheduledApex') AND status IN ('Holding', 'Queued', 'Processing', 'Preparing') AND ApexClass.Name IN ('batchProcessor') AND ApexClass.NameSpacePrefix = 'AAkPay')” into the query
- Then press Execute
- Double click into the Id Cell. Copy the ID.
- NOTE, you will need to repeat Step Execute code to Abort Phantom Job for each row. NOTE Press the Execute (step 3) button each time Batch Processor Id's might change in the time it takes you to do the other steps.
SELECT Id,Status,JobType,ApexClass.Name,CreatedDate,LastProcessed,JobItemsProcessed,NumberOfErrors FROM AsyncApexJob WHERE (JobType IN ('BatchApex','ScheduledApex') AND status IN ('Holding', 'Queued', 'Processing', 'Preparing') AND ApexClass.Name IN ('batchProcessor') AND ApexClass.NameSpacePrefix = 'AAkPay')
3.5. Developer Workbench
3.5.1. Login to Developer Workbench
- Open another tab and paste “https://workbench.developerforce.com/execute.php” into the URL
- Accept Terms and Login
3.5.2. Select API Version 31
Click on the top link with the user + company + version No.
Select Version 31
3.5.3. Run Apex Code to Abort Phantom Jobs
- Select "Apex Execute" from the Utilities Menu
- Press Next button
3.5.3.1. Execute code to Abort Phantom Job
- Copy “system.abortjob('7074Q00007LyJwe'); “
- Change “7074Q00007LyJwe” to the id that was copied from Step 3 above.
- Press Execute
- Repeat for each Id in Step Run query on Batch Processors.
NOTE, you should press the EXECUTE button (as in the Run query on Batch Processors step) again as the Id's might have changed in the time you've taken to do these updates.
Continue to repeat until there are no Id's left from the query.
system.abortjob('7074Q00007LyJwe');
3.6. Restart Batch Processors
See procedure How do I start or stop the Batch Processor on how to START the Processors
The owner on the Payment Txn has the correct access, what is causing this?
1. Check the Payment Form used by the Payment Txn. see which user is in the Donor Care field
2. On the user, give them the Payments2Us Admin permission set. See Step No. 5 here on how to assign permission sets.
5.1. Check for ApexToken Batch Jobs
ApexToken jobs are internal jobs used by the Salesforce Platform to manage flex queues. See: https://help.salesforce.com/articleView?id=000338931&type=1&mode=1 for more information.
On occasions, these can become stuck and cause other jobs (including Payments2Us ones) to also remain on hold.
The below sections provide you with all the information you need. Should Salesforce support come back with further questions, or if you need more information, then you will also need to have Payments2Us Premium support.
This issue is a Salesforce platform one and should NOT require you to have Salesforce developer support. If Salesforce support insists on this, we suggest you contact your Account Manager to escalate, or point support to the following bulletin board articles where this was previously resolved.
- https://developer.salesforce.com/forums/ForumsMain?id=9060G000000I4EzQAK
- https://salesforce.stackexchange.com/questions/207445/new-apex-batch-jobs-always-end-up-in-the-flex-queue-in-holding
Read through the information below in this article before contacting Salesforce support as it with locate the information you need and if this is actually the issue that is causing your problems.
5.2. Open the Developer Console and check for stuck ApexToken Jobs
5.2.1. Open the Developer Console
- Top right click on the settings cog
- Followed by Developer Console
5.2.2. Run query on ApexToken Jobs
- Click on the query tab.
- Paste “SELECT Id, JobType, ApexClass.Name,CreatedDate, Status FROM AsyncApexJob WHERE JobType = 'ApexToken'” into the query
- Then press Execute
You'll need a screenshot similar to the below when logging a Salesforce support request.
SELECT Id, JobType, ApexClass.Name,CreatedDate, Status FROM AsyncApexJob WHERE JobType = 'ApexToken'5.2.3. Determine if you have an ApexToken issue
In the screenshot in step "Run query on ApexToken Jobs" above, for item (4), you can see that there are two jobs that are at a "Queued" status since 2016. If there is ONE or more jobs that have a queued date this is old (more than a day or so), then this indicates you have this particular issue. If that is the case, then continue with the next steps
5.3. Locate jobs being held up
This step will be required when submitting the case to Salesforce support. They are likely to asked you for an example of a job being held up.
5.3.1. Navigate to Apex Jobs
From setup (Cog - top right hand corner). Search for "Apex Job" and click into the menu option.
5.3.2. Add a filter to show queued or running jobs
Click the Create New View link that is next to the View picklist filter options
- Give the filter a name, e.g. "All Running Jobs"
- Add a filter based on the Status equals Queued,Processing,Preparing,Holding
- Press the SAVE As button
5.3.3. Locate Jobs that are held
If you see jobs that have a "Submitted Date/Time" that was quite a while ago. In particular if you see "batchProcessor" (Apex Class) and the Submitted Date/time was more than an hour ago, then that is an example of a job affected by the ApexToken held job issue. You will need info when contacting Salesforce support.
You'll need a screenshot similar to the below when logging a Salesforce support request.
5.4. Logging a support case with Salesforce
The ONLY way this issue can be fixed is through Salesforce support/developer support.
Log a case along the following lines.
Subject: "Apex Batch Jobs hanging caused by ApexToken jobs stuck in Hold status"
Description:
Dear support,
We have a platform issue whereby ApexJobs are being held up by stuck/held ApexToken Jobs.
We understand from bulletin boards that this is going to require this case to be escalated. This is as noted at:
• https://developer.salesforce.com/forums/ForumsMain?id=9060G000000I4EzQAK
We have observed the stuck Apex Jobs (Copy/attached screenshots from step "Run query on ApexToken Jobs")
This is causing our Apex Jobs to become stuck (copy/attach screenshots from step "Locate Jobs that are held")
The above is a known Apex Flex Queue issue and is NOT A DEVELOPER SUPPORT ISSUE. We understand the Developer Support Team from Salesforce may need to get involved in fixing the Salesforce platform issue though.
The result of the Batch Processor running is the Matching Process is performed to link to existing Accounts/Contacts or to create new ones.
To avoid possible issues, the processor checks to see if the group of transactions it is trying to process might have possible duplicate Contacts. If it does, it skips processing these and will then pick those for processing in the next run (approx 10 minutes time).
The duplicates within the group of Payment Txns include:
- Membership No
- Fundraiser Id
- Reference
- Mobile Phone
- Contact Id
- Account Subscription or Contact level Subscription
- First/Last Name
In the Apex Jobs, the Batch Processor is constantly being aborted/submitted. The reasons for this are:
- Salesforce ONLY provides a method of running a batch processor daily. Payments2Us need this to run more frequently.
- Payments2Us schedule for 10 minutes time. The Batch Processor starts in 10 minutes.
- Then instead of waiting a whole day, Payments2Us cancel the job and then reschedules to run again in the next 10 minutes time.
- The above means Payments2Us can effectively run the batch processor every 10 minutes, vs. daily.
Sometimes, what can happen is Salesforce delays the batch process due to maintenance, server loads etc. and the sequence of this can go out and then the possibility of 2 run. When this happens, you need to STOP, then re-START the Batch Processor.
When a new Merchant Facility is created or the "Print Email Receipt" is changed on the Primary and Active Merchant Facility, then a background processor called "contactUpdateProcessor" runs.
This process copies the value in the field "Print Email Receipt" on the Merchant Facility to all Contacts, field - "Print Email Receipt".
This is required as some Workflows/Flows will send Receipts/Reminders/Statements to the Contact. If the Contact does NOT have an email, then the Print Email Receipt will have an internal users email address. The internal person receiving the email will then print of the document and mail out.
NOTE:
The processor updates contact records. This may update Contacts have have invalid data or validation rules. This means the updates are not successful and the Payments2Us Error Log entry is created with the error.
If you were to do an edit/save on the Contact record, you would very likely receive the same error message - I.e. This is a data issue in your system and not Payments2Us related.
Please check the following:
- "Error Log - Payments2us". There might be an error in here that describes an issues. This is likely to be a restricted Picklist, Validation Rule, Flow/Process Builder issue (NOTE: Issues caused by these are outside the scope of Payments2Us Standard support)
- The Batch Processor might need to be stopped/started.
- Receipting might be set to Manual or Similar. See Checkout FAQ for Payment Txns being stuck.
- The user that started the Batch Process might not have a Payments2Us Permission Set assigned.