Today, I am going to share a code snippet that I found to be the most effective in handling failures when adding multiple users using the "Add Multiple Subscribers" Sitecore Send API. The failure occurs due to the API's rate limit of 2 requests per 10 seconds.
Adding multiple subscribers to a mailing list is essential for email marketing and automation when working with the Sitecore Send API. However, the API has rate constraints, and it returns a 429 Too Many Requests error if too many requests are made in a short amount of time, i.e., more than 2 requests within 10 seconds. If it is not handled correctly, there will be unreliability in the subscriber addition procedure, leading to unsuccessful requests, which might result in some users not being added to the mailing list.
Understanding the 429 Error in Sitecore Send API
The 429 error indicates that the API has received too many requests in a given timeframe. Ideally, an API should return the response with an HTTP 429 status code, along with a Retry-After header specifying how long to wait before making another request. However, in Sitecore Send’s case, even when rate limiting occurs, the API still returns an HTTP 200 status code, embedding the 429 code within the response body (i.e., the response is 429).
Handling Rate Limits Effectively
In order to ensure a seamless subscriber import procedure, it is important for developers to follow the points below:
1. To detect 429 errors, look at the body of the API response rather than just the HTTP status code, as in the Sitecore Send API, the HTTP status code returned is 200, but the "Code" in the response is 429.
2. If available, get the Retry-After value from the response headers. However, this is not available in the Sitecore Send API since the HTTP status code returned is 200.
3. Implement a retry system that avoids excessive retries and maintains rate constraints by introducing delays.
C# Implementation to Handle 429 Error Effectively for Sitecore Send
The following C# method efficiently handles rate limits when adding multiple subscribers to a mailing list using the "Add Multiple Subscribers" Sitecore Send API. It retries failed requests based on the Retry-After header (if available, otherwise it retries the API call after 5 seconds) while maintaining efficiency and avoiding unnecessary failures.
// Method for Adding multiple users to a mailing list with rate limit handling
public AddMultipleUsersResult AddMultipleUsersHandlingRateLimit(List<string> userNames, string mailListID, SitecoreSendSettings settings, int maxRetries = 5)
{
// Prepare the request payload with user emails
var addMultipleUser = new AddMultipleSubscribers.AddMultipleUser
{
Subscribers = userNames.Select(user => new AddMultipleSubscribers.UserInfo { Email = user }).ToList(),
HasExternalDoubleOptIn = false
};
// Serialize the request payload to JSON
string jsonData = JsonConvert.SerializeObject(addMultipleUser);
int attempt = 0;
// Retry loop for handling rate limits
while (attempt < maxRetries)
{
try
{
// Create the HTTP POST request
var httpWebRequest = (HttpWebRequest)WebRequest.Create($"{settings.BaseURL}/subscribers/{mailListID}/subscribe_many.json?apiKey={settings.APIKey}");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
// Write the JSON payload to the request body
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(jsonData);
streamWriter.Flush();
}
// Send the request and read the response
using (var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse())
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
var sitecoreSendResult = JsonConvert.DeserializeObject<AddMultipleUsersResult>(result);
// Check if rate limit (HTTP 200) was hit but "Code" value is 429
if (sitecoreSendResult?.Code == 429)
{
int retryAfter = GetRetryAfter(httpResponse);
Sitecore.Diagnostics.Log.Info($"Rate limit hit for adding multiple users. Retrying after {retryAfter} seconds.", typeof(SitecoreSendHelperMethods));
System.Threading.Thread.Sleep(retryAfter * 1000);
attempt++;
continue;
}
// Successful response (HTTP 200) return result with "Code" value as 0
return sitecoreSendResult;
}
}
catch (Exception ex)
{
// Log any exceptions and rethrow
Sitecore.Diagnostics.Log.Error("Request failed with error: " + ex.Message, ex, typeof(SitecoreSendHelperMethods));
throw;
}
}
// Throw exception if max retries exceeded
throw new Exception("Max retry attempts reached. Unable to complete the request to add multiple users.");
}
// Helper method to extract 'Retry-After' header value or default to 5 seconds
private int GetRetryAfter(HttpWebResponse response)
{
if (response.Headers["Retry-After"] != null && int.TryParse(response.Headers["Retry-After"], out int retryAfter))
{
return retryAfter;
}
return 5;
}Reason Behind This Implementation
1. Instead of relying only on the HTTP status code, it checks the "Code" in the response for a 429 error.
2. It dynamically waits for the appropriate Retry-After time before retrying to avoid unnecessary failures. If the Retry-After header is not available in the response, it sets a 5-second delay to reduce API call failures.
3. Uses using blocks for StreamWriter, StreamReader, and HttpWebResponse to prevent memory leaks.
4. Since there is a retry mechanism with attempts, the chances of missing users being added to the mailing list are negligible.
Handling rate limits effectively is crucial when working with APIs like Sitecore Send. By implementing a robust retry mechanism based on the Retry-After header and response body codes, you can ensure a seamless and reliable subscriber import process. This approach helps prevent failures and ensures compliance with API rate limits, ultimately improving your email marketing automation experience.
References
Add Multiple Subscribers - https://doc.sitecore.com/send/en/developers/api-documentation/add-multiple-subscribers.html
API Rate Limiting - https://doc.sitecore.com/send/en/developers/api-documentation/api-rate-limiting.html
That's All for Today,
Happy Coding
Coders for Life