Salesforce Limitations and Drawbacks
In the last post, we covered how to use Salesforce (SF) to embrace the chaotic and Excel-driven order management process in Manufaktura. We saw how SF can automate lots of manual jobs and reduce the amount of error handling by taking care of data integrity. But from both a business and tech point of view, SF alone is not enough to constitute a speed-oriented software architecture. In this article, we’ll show you why and then how to use SF API to connect your IT components into a well-oiled platform.
The API
It’s a little-known fact that SF, a $50 billion company, generates 50% of its revenue through its API. There has to be something in it, right? In the following chapters, we’ll try to explain what makes it so popular among businesses and how an early stage business like “Manufaktura” can use it to scale their operations.
In fact, the REST API is going to be a core concept of Manufaktura software architecture and thus of further articles in the series. It will become the main pillar of the framework that Manufaktura is built on. A leverage which allows the Manufaktura team to deliver business results more quickly while simultaneously saving enormous amounts of engineering time.
But before we use the SF REST API, let’s see how you can adjust your processes system without tapping into external services. This should give you more knowledge about what can be done within SF and its internal programming environment.
The first problem of scale
The problem the team stumbled upon is the outburst of initial not-so-successful relationships. One of the manufacturers didn’t manage to meet the designer’s expectations and so, the latter didn’t want to work with them anymore. The situation occurred a couple of times for multiple designer - manufacturer pairs and, therefore, remembering which manufacturer should be excluded from the tender became problematic. The dev team was given a task to automate this.
Fortunately, this issue can be quickly solved with an SF Apex code. All we need to do is go to the ManufacturerSelectJob, you can find it in our previous article, and add a new business rule.
The feedback will be collected by customer service agents after an order is finished/completed and they will enter the data into a newly created Order’s field - for a while we will keep data entry as manual.
So, before we change the code, we have to modify the related SF objects accordingly.
1. Add a picklist field "Feedback" to the "Order" object with the following values: POSITIVE, NEGATIVE.
2. Add hidden formula field called "Feedback Filter" to "Order" object which will be used in the query, it will concat Account Id and Manufacturer Id.
3. Add a hidden formula field called "Feedback Filter" to the "Order Proposal" object that will be used in the query, it will concat Account Id (from Order) and Manufacturer Id (Introduced to work around the SOQL limit - direct field to field comparison is not supported in SF, more info).
4. Remove the scheduled job to allow code modification (Setup ->Environments/Jobs/Scheduled Jobs).
5. Edit ManufacturerSelectJob: (modified lines start with the “+”).
{{CODE}}
global class ManufacturerSelectJob implements Schedulable {
global void execute(SchedulableContext SC) {
+ List<Order> ordersWithNegativeFeedback =
+ [ SELECT feedback_filter__c
+ FROM Order
+ WHERE feedback__c = 'NEGATIVE'
+ AND feedback_filter__c != null ];
+
+ List<String> negativeFeedbackPairs = new List<String>();
+ for (Order order : ordersWithNegativeFeedback) {
+ negativeFeedbackPairs.add(order.feedback_filter__c);
+ }
List<order_proposal__c> acceptedProposals =
[ SELECT Id, manufacturer__c, order__c
FROM order_proposal__c
WHERE order__r.EffectiveDate < TODAY
AND order__r.Status = 'AWAITING PROPOSALS'
+ AND feedback_filter__c NOT IN :negativeFeedbackPairs
ORDER BY order__c ASC, price__c ASC ];
List<Order> acceptedOrders = new List<Order>();
Set<String> acceptedOrderIds = new Set<String>();
for (order_proposal__c acceptedOrderProposal : acceptedProposals) {
if (acceptedOrderIds.contains(acceptedOrderProposal.order__c)) { continue; }
acceptedOrderIds.add(acceptedOrderProposal.order__c);
Order acceptedOrder = new Order();
acceptedOrder.Id = acceptedOrderProposal.order__c;
acceptedOrder.proposal__c = acceptedOrderProposal.Id;
acceptedOrder.manufacturer__c = acceptedOrderProposal.manufacturer__c;
acceptedOrder.manufacturer_selected_date__c = System.now();
acceptedOrder.Status = 'MANUFACTURER SELECTED';
acceptedOrders.add(acceptedOrder);
}
update acceptedOrders;
List<Order> cancelledOrders =
[ SELECT Id, Status
FROM Order
WHERE EffectiveDate < TODAY
AND Status = 'AWAITING PROPOSALS'
AND Id NOT IN (SELECT order__c
FROM order_proposal__c
+ WHERE feedback_filter__c NOT IN :negativeFeedbackPairs) ];
for (Order cancelledOrder : cancelledOrders) {
cancelledOrder.Status = 'CANCELLED NO PROPOSALS';
}
update cancelledOrders;
}
}
{{ENDCODE}}
6. Re-schedule the job.
Quick, right? We improved an important feature and therefore a whole process within 30 minutes. Plus, you can immediately generate more value out of this change by creating a report which sums up manufacturers’ performance:
You got the flow? A quarter of configuring and scripting here and there and bam, the business gets value. And this doesn’t require us to tap into external databases or other systems, we can store and maintain data structures like this one easily within SF objects.
What’s most important is, within an hour or so, we helped the operations team reduce the number of painful assignment mistakes which, when accumulated, might cause a lot of harm to Manufaktura’s trustworthiness.
But as you saw in the 3rd point above (the workaround), SF and its limitations don’t always allow for such a seamless experience.
When to go outside Salesforce
There are 2 main reasons for leaving the SF environment when it comes to implementing the business logic: SF’s (too) strict limits and poor developer experience.
Limits
SF’s limits are an issue in itself. They need a 46-page cheat sheet to explain how they control and charge their users. This includes number of objects, file sizes, but also access to features, execution time of jobs, and the number of REST API calls. This, of course, varies depending on the plan you chose. The bigger the number of user licenses you bought, the better limits you get. But there are no official pricing lists, you have to negotiate it with a sales rep.
Sometimes you can purchase an extra pack, sometimes the limit is fixed and you have to work it around. If the latter is true or if paying more isn’t an option, you can transfer the responsibility of storing/processing/calculating to an external system, either your bespoke module or a 3rd party tool.
Dev experience
In the recent Stack Overflow Developer Survey, SF was rated the second Most Dreaded Tech Platform of the Year. Sharepoint won, but only by a nose. There are a couple of reasons behind this infamous place in the rankings. Paraphrasing some of the issues put down by Jeremy Kraybill:
- Poor source control integration.
- No continuous integration.
- Long deployments:
- You can’t add a new code to production immediately, you have to wait up to 30 minutes.
- Poor support for multi-developer/multi-team environmentsToo many platform limits (see above)No local execution emulator that doesn’t require server round trips of saving Apex code (this has changed with DX, see below).
- Limited Java API exposure in Apex. No binary file manipulation, image/sound APIs, or utility APIs.
This translates into a cumbersome developer experience (DX). Devs can’t get into their flow like they can with different technologies. The platform reminds them of its limitations on every corner. And poor DX translates into drops in motivation and less productivity in the end.
The good news is that SF listened to the community and just released Salesforce DX - a new tooling that addresses some of these issues. Is their long-standing motto “No software” finally going to fade away? :)
Lastly, if you want to lessen your source code’s reliance on proprietary language (or you’re having problems finding devs with the SF skill set) and have to work around SF’s limits, you should consider implementing and integrating an accompanying system. Let’s see how you can do it so that it doesn’t mess with our speed-oriented software development methodology.
How to connect SF to the outside world?
Input
Let’s take more of the burden off the customer service team’s shoulders. We will replace the manual data entry of new orders with an online form. We have 2 options here:
- quickly sketch up a custom website which connects to the SF API through JS Force or another SDK, or
- use a form generator and add new records through Zapier.
To speed things up, we’ll take the 2nd path. Let’s employ Typeform which is a popular online form builder. As you might’ve guessed, the first step is to create a simple form like this one: https://bandro.typeform.com/to/q2jhFL
Next, we need to send the completed forms to SF. As both SF and Typeform are on the Zapier store, the configuration is straightforward. You just need to hook on a new order and create respective Account and Order objects.
Notifying external components
Now, let’s see how we arrange SF -> other services communication. We will work around one the most rigid limits of the SF API - email send out. SF gives you 5k emails per day, it also limits the number of “To:” addresses to 100 per email. Moreover, there’s no email deliverability report available, which you might be interested in during the early days of the company. It’s reasonable then to resort to other services like Mandrill or SendGrid.
In this section, we’ll show you how to send automatic notifications when there’s a change of state in one of the SF objects. Let’s create a trigger which will push a JSON payload to an external API.
1. Go to Setup -> RemoteSiteSettings and whitelist a remote address you want to connect to
2. Create a class, e.g. BackendService, which is responsible for creating an HTTP request, @future(callout=true) annotation makes the request async
{{CODE}}
global class BackendService {
@future(callout=true)
public static void send(string endpoint, string payload) {
try {
Http http = new Http();
HttpRequest request = new HttpRequest();
request.setEndpoint(endpoint);
request.setMethod('POST');
request.setHeader('Content-Type', 'application/json');
request.setTimeout(120000);
if (payload != null) {
request.setHeader('Content-Length', '' + payload.length());
request.setBody(payload);
} else {
request.setHeader('Content-Length', '0');
request.setBody('');
}
HttpResponse response = http.send(request);
if (response.getStatusCode() >= 300) {
// Error handling
}
} catch(Exception error) {
// Error handling
}
}
}
{{ENDCODE}}
3. In Setup -> Apex Triggers, add a simple trigger watching Orders. It compares order status and invokes the BackendService, passing the relevant objects as JSON.
{{CODE}}
trigger OrderStatusTrigger on Order (after insert, after update) {
if (Trigger.isAfter) {
for (Id orderId : Trigger.newMap.keySet()) {
Order newOrderData = Trigger.newMap.get(orderId);
Order oldOrderData = null;
if (Trigger.isUpdate) {
oldOrderData = Trigger.oldMap.get(orderId);
}
if (oldOrderData == null || oldOrderData.Status != newOrderData.Status) {
string endPoint = 'https://manufaktura-sandbox.herokuapp.com/order/state/' + newOrderData.Status.trim().replace(' ', '-').toLowerCase();
Map<string, object> payload = new Map<string, object> {
'order' => newOrderData,
'account' => (Account)DTOUtil.QueryForSingle('Account', ' ac WHERE ac.Id = \'' + newOrderData.AccountId + '\'')
};
BackendService.send(endpoint, JSON.serialize(payload));
}
}
}
}
{{ENDCODE}}
4. Implement a service consuming the request, example in Node.js.
{{CODE}}
var Logger = require("./../../../../logger");
var email_service = require("./../../../../services/email");
var logger = new Logger({ prefix: "order/state/pending-validation" });
module.exports = function(request, response) {
var order = request.body.order;
var account = request.body.account;
if (!order || !order.Id || !account || !account.Id) {
logger.error("Order or Account is not defined | Data: %j", order.body);
return response.status(200).end();
}
logger.info("Order: %s Account: %s", order.Id, account.Id);
response.status(200).end();
email_service.send({
"from": {
"email" : "bandro@rspective.pl",
"name" : "Manufaktura Team"
},
"personalizations": [{
"substitutions": {
"[[customerName]]" : account.Name || "",
"[[orderNumber]]" : order.OrderNumber || ""
},
"to": [{
"email" : account.email__c || "team@voucherify.io",
"name" : account.Name
}]
}],
"subject": "Welcome",
"template_id": "f534796e-c338-4f01-9242-bfa7ec070b38"
})
.then(result => {
logger.info("Email sent - Order: %s Account: %s Result: %j", order.Id, account.Id, result);
})
.catch(error => {
logger.error("Email not sent - Order: %s Account: %s Message: %s Error: %j Stack: %j", order.Id, account.Id, error, error, error && error.stack);
});
};
{{ENDCODE}}
Again, a couple of hours of a single developer’s time and the important feature got automated.
Data analytics unleashed
When the traffic rises to millions of requests and the SF API becomes too slow or too costly, you can always resort to Heroku Connect (HC). This platform allows you to mirror an SF database as a regular Postgres instance. In this way, you can point data-sensitive applications to the follower database.
HC gives you 10k records for free right after you install the addon. If you want more than this, you should contact your SF sales rep and negotiate. There are no clear rules here, but the more user licenses you buy, the more HC records you get.
How it works - you need to buy a database in the Heroku marketplace, add the HC addon, and finally map which objects should be synchronized. HC allows you to sync objects in the range <10, 60> minutes. What you get as a result is the ability to run SQL-based reports without utilizing any SF limits, e.g. you can connect one of the popular BI tools like tableau.
What can you configure about synchronization:
- Which objects.
- Which fields.
- Poll frequency.
Moreover, HC supports bi-directional synchronization which means that we can modify data within the postgres instance and the changes will be automatically applied on SF records. Speaking of synchronization, I should also say that HC takes care of metadata changes propagation. If you remove a field in SF, it disappears in the postgres table too. Note: SF is always the master of data.
Surprisingly, every call to SF from HC doesn’t utilize the daily API limit either! This is then a perfect tool to build data exposing UI applications.
Although HC is a mature and powerful tool, it’s still under development. New features come, and some go - they’ve recently withdrawn support for formula or roll-up summary type. So, consult the docs before you decide to give it a chance. Also, when it comes to pricing, here’s an interesting fact: the negotiated limits are not strict and HC won’t block you when you exceed them. All in all, you should discuss it with your account manager again.
Scaling & pricing
Lastly, we’d like to share some pricing guidelines/watchouts which help you prepare your SF billing forecast and perhaps decide which functionalities might be taken out to external service.
Let’s start with the list of limits you should be aware of:
- 15k API calls for starter.
- Strict email limits.
- Maximum amount of code used by all Apex code in an org - 3MB, see other code-related limits.
- Data/file storage - e.g. data storage is 1GB per one user license.
- Number of custom reports.
- 4000 characters for a db query and other SOQL limits.
Studying the limits and their impact on the bill is a key exercise to be done when deciding if something should be coded using SF objects and APEX or outsourced to your bespoke system via the API.
Summary
Within a day or so, we managed to implement 2 crucial features for Manufaktura. Step by step, our platform is automating more and more processes, allowing customer service, marketing, and the operations team to take care of other hot topics in the fast-growing company.
We’ve also learned when and how to move outside SF (with REST API and Heroku Connect) to do the job in the relevant/favorite technology or/and without hitting SF limits.
The small codebase and flexible architecture allow us to introduce changes super-fast and scale. On this basis, we can now jump into automating subsequent department processes. Almost, because there’s one thing which doesn’t scale and it’s gonna hit us when new features are released fast and pile up - it’s the error handling. In the next post, we’ll suggest a solution which helps you build a safety net which is tight enough to keep the business running and lean enough to be maintainable when your source code base starts to swell.
.jpeg)
.jpeg)
.jpeg)