Salesforce provides a number of APIs, or Application Programming Interfaces, to integrate and work with data, each with their own use case. Today we’re going to look at Bulk API – a programmatic solution for working with large batches in Salesforce. But just how much data are we looking at that would justify the use of Bulk APIs in Salesforce? And with two versions of the Bulk API available, which one is best to use? Let’s take a look at when you should use Bulk APIs in Salesforce.
Bulk API – An Overview
The Bulk API is one of several APIs that Salesforce provides out-of-the-box. It’s a programmatic, REST-based API solution that allows you to work with large data sets in Salesforce. This process works asynchronously, meaning that it will work concurrently with other data operations in your org.
When Should You Use Bulk APIs?
So how much data is enough to justify the use of Bulk APIs in Salesforce? According to the documentation, any data operation, including uploading, querying, and deleting more than 2,000 records should make use of the Bulk API. If you’re working with under 2,000 records, it’s recommended to use bulkified synchronous operations with a different API.
While Bulk APIs enable you to work with large data sets asynchronously, that does not absolve it from adhering to governor limits. In general, you can submit up to 15,000 batch jobs per rolling 24-hour period. Additionally, jobs that are older than seven days are either deleted or cleaned up, based on their status. And the maximum time that a job can remain open is 24 hours. Limits may also vary based on the type of job (ingest, query, etc) and whether a Bulk API or Bulk API 2.0 are used. For a more detailed overview, check out the Bulk API Limits doc from the Salesforce documentation.
When Should You Use Bulk API vs Bulk API 2.0?
Salesforce has two versions of this API – Bulk API and Bulk API 2.0. Bulk API 2.0 is a new and improved version of Bulk API which includes its own interface. While Bulk API 2.0 is a newer version of Bulk API, both are still available to use, each with their own set of limits and features.
The main difference between Bulk API and Bulk API 2.0 is how it processes batches. With Bulk API, the query submitted for processing is first batched before it is processed, thus resulting in an iteration for each one. But with Bulk API 2.0, this extra step is eliminated as it instead has a more optimized process.
Bulk API Process
Bulk API 2.0 Process
Given the optimized process and interface that comes with Bulk API 2.0, sticking with the newer version may seem like a clear choice. However, based on your use case, it may be in your best interest to use the original. Here are some considerations to keep in mind when deciding which one to use:
Authentication
Bulk API 2.0 supports all OAuth 2.0 flows supported by other Salesforce REST APIs. Bulk API requires a special X-SFDC-Session header fetched with SOAP API’s login() call.
Ingest Data Format
Bulk API 2.0 only supports CSV format. In addition to CSV format, Bulk API supports XML, JSON, and binary attachment processing.
Limits
Because of the differences in processing for both APIs, Bulk API’s limits are processed based on batches per day, rather than records per day.
Data Loader
Bulk API supports Data Loader, while Bulk API 2.0 does not.
How Bulk Jobs Are Processed
Bulk API
With Bulk API, you create a job containing one or more batches, specifying the type and operation. The batches are then sent to the server via a POST request, whose status can be tracked via the JobInfo resource. Once this is complete, the batch processing begins and the status can be checked via the BatchInfo resource. Once the batch is complete, each batch’s results are returned in a result set for each.
This job can be aborted at any time in the process; however, any already completed batches cannot be rolled back.
Bulk API 2.0
When you create a job in Bulk API 2.0, Salesforce will automatically optimize your job before processing in an effort to complete the job as soon as possible without timeouts or failures. Using the new interface, you can check the status of your job by navigating to Setup > Bulk Data Load Jobs.
When working with ingest jobs, Salesforce will automatically break the data into 10,000 records each. With query jobs, it will break the data into chunks of 100,000 to 250,000 records.
How to Monitor the Status of Bulk API Jobs
While jobs are processing, you can check in on the status of jobs by navigating to Setup > Bulk Data Load Jobs. Clicking on a job by its Job ID will show more details about that job, including the following values listed in the Salesforce documentation.
I hope this information helps you to better understand when you should use Bulk APIs in Salesforce. If you have any additional questions, feel free to reach out!
Julie Anna Contino
Jr. Developer
Julie Anna is a junior developer with a passion for learning and problem-solving. She graduated with a Bachelor's degree in Computer Science and has four years of development experience. She's excited to be a part of the Salesforce ecosystem and combine her previous experience with her passion for helping clients thrive.
About Roycon
We’re an Austin-based Salesforce Consulting Partner, with a passion and belief that the Salesforce platform’s capabilities can help businesses run more efficiently and effectively. Whether you are just getting started with Salesforce or looking to realize its full potential, Roycon specializes in Salesforce Implementations, Salesforce Ongoing Support, and Salesforce Integrations, and Development. We’re the certified partner to guide the way to increase Salesforce Adoption, make strategic decisions, and build your Salesforce Roadmap for success.