Slides du Paris Salesforce Developer Group de Mai 2017.

1. Creative way of coding
Jean-Luc Antoine, @ApexAPI, May 2017
Think out of the box for more
efficient Salesforce
administration
Paris Salesforce Developer Group
1

2. Use Case
• As a Salesforce administrator, I want to identify the fields that
have not been used since a long time.
• My object is
- hosting millions of records
- heavily customized
• I want to do this for multiple objects
2

3. Constraints
• I need to
- Optimize the network
I don’t want to download gigabytes of data over the network
- Optimize the API
I have limited calls per 24h
- Automate the process
I will not do this manually, it has to be automated
- Optimize duration
I should not wait too much to get the information
3

4. Quick facts
• Large Date Volume orgs are difficult to query
- Timeouts can occur frequently
- Can hit the limit of long running queries
• The way you are using the API can impact perf
- HTTP1.1 / keepAlive
- Gzip compression
This is out of scope of this study
4

5. Use case - reducing the scope
• We will not analyze
- Formula fields
- Non nilable fields (checked as required)
- Checkboxes (true or false, no capability to identify if it has been set)
- System fields
• We must configure
- The Date field to be used
CreatedDate or LastModifiedDate? Impact on migration on those fields?
5

6. Solution 1 – download + process
• Use any tool such as Dataloader
• Parse locally the resulting CSV file
• Impacts
+ Moderate API usage
-- Long time to prepare the file on server side
-- Long time to download the file
-- Downloading everything even if the first records tells you all fields are
used.
Waste of resource and time.
6

7. Solution 1 – duration
7
Duration
API usage
Solution 1
Network usage
Solution 1
End of
processing 1
Qty
•Run the query
•Prepare the resulting file
for download
•Api calls to identify when
it is ready for download
Download
Local
processing
•Not efficient if all
the fields have been
recently used as we
still download
everything

8. Solution 2 – API + continuous process
• Use the API to run your query
• Do a queryMore() until you find the last used date for all fields
- You receive your records in bulk of BatchSize
- The more you query fields, the smaller your BatchSize is
50 fields can drive to a batchsize of 200 while 3 fields can give you 2000 records in a
batch
• Impacts
+ Optimizing the number of records retrieved as you can stop the process as
soon as you have the information for all your fields
-- Lots of API consumption (small batch size, huge number of records)
-- Long duration because lots of API round trips + long initial query
8

9. Solution 2 - duration
9
Duration
API usage Solution 2
Network usage Solution 2
End of
processing 2
Qty
Server time
before first
result
N x
(processing +
queryMore())
•Can have timeouts
•Slow when all fields
have been recently
used
•Not efficient for the
last fields to be found
as we retrieve all
fields

10. Solutions - duration
10
Duration
API usage Solution 2
Network usage Solution 2API usage
Solution 1
Network usage
Solution 1
End of
processing 1
End of
processing 2
Qty

11. Solution 3 – refined query
• Use the API to run your query
• Do a queryMore() until you find the last used date for all fields, or
do a new query() with less fields if you found some fields
- The BatchSize will increase progressively
The quantity of records per roundtrip will increase
The total duration will decrease compared to Solution 2
• Impacts
+ Optimizing the number of records retrieved as you can stop the process as
soon as you have the information for all your fields
+ Less API calls than Solution 2
+ Quicker compared to Solution 2 as you increase the throughput
-- Still long initial query, and some when refining the query
11

12. Solution 3 - duration
12
Duration
API usage Solution 3
Network usage Solution 3
End of
processing 3
Qty
•Multiple queries that
are progressively faster
•Accelerating throughput,
reducing the time

13. Solutions - duration
13
Duration
API usage Solution 2
Network usage Solution 2API usage
Solution 1
Network usage
Solution 1
End of
processing 1
End of
processing 2
Qty
API 3
Network 3
End 3

14. Solution 4 – server processing
• Same query optimization as Solution 3
• Executed as ApexAnonymous on the Server
• Impacts
+ Capability to process much more records in 1 roundtrip
Theorically up to 50k, but limited to CPU time less than 10k
+ Immediate query because of the “limit 10k”
No need to wait for the snapshot on the DB server
+ Optimized network
Only results are transmitted, not the data
+ Optimized API
Because of the huge batch size
14

15. Technical tip to develop Solution 4
• Use ApexAnonymous
- No need to deploy/inject anything in the org
- executeanonymous() from SOAP API
Very efficient, retrieve Apex debug logs as the result
- executeAnonymous using the tooling API
Requires lots of API calls
1. POST to /tooling/sobjects/traceFlag to define the debug log level
2. Execute the anonymous Apex code
3. Get the Id of the last log from the ApexLog tooling object
4. retrieve the debug log content by querying the ApexLog object Body
15

16. Technical tip to develop Solution 4
• How to get the result of Apex processing on client side?
- The Apex Code needs to prepare a JSON dump using system.debug()
- The client need to retrieve the debug logs, filter the custom debug
statements to regenerate the JSON, then use it
• Apex Code is dynamically autogenerated to have the progressive
query enhancement
- Removing progressively from the query the fields that have been found
16

17. Solution 4 - duration
17
Duration
API usage
Solution 4
Network usage
Solution 4
End of
processing 4
Qty
•Immediate query
performance
•No timeout •Accelerating throughput,
reducing the time

18. Solutions - duration
18
Duration
API usage Solution 2
Network usage Solution 2
API 1
Network usage
Solution 1
End of
processing 1
End of
processing 2
Qty
API 3
Network 3
End 3
End 4
Net4
API 4

19. Code
• Available to run directly on AdminBooster Salesforce Playground
• Solution 3
https://www.adminbooster.com/tool#listing=ajaxlastused
• Solution 4
https://www.adminbooster.com/tool#listing=ajaxlastusedapex
19

20. Q&A