BI Team @ LinkedIn hosted a user group meeting for MicroStrategy customers in bay area. Presentation includes information about LinkedIn, concepts of metadata driven model for business dashboards, customizations using SDK, JSP and JQUERY.

1. SF MicroStrategy Bay Area Meetup
December 4, 2013
LinkedIn Confidential ©2013 All Rights Reserved

2. Agenda
• About LinkedIn
• BI Challenges
• Administration Hacks
• Q&A

3. Our Mission
Connect the world’s professionals to make them
more productive and successful.
Our Vision
Create economic opportunity for every
professional in the world.
Members First!

4. The world’s largest professional network
Over 65% of members are now international
19
Languages
259M+
Members Worldwide
Growing at more than two members per second
>91%
Fortune 100 Companies
use LinkedIn Talent Soln to hire
>3M
Active Company Pages
>30M
Students
©2013 LinkedIn Corporation. All Rights Reserved.

5. Technology Stack

6. Technology Stack

7. Internal User Groups
Product
Biz-Ops
Data Analytics
Infrastructure
Hiring Solutions
Data
Sciences
Executives
Marketing Solutions
Talent Solutions

8. Business Intelligence Team
Bhavani Akunuri
Kevin Simon
Poorna Lankipalli
Sandeep Deviah
Sunil Shirguppi
Qun Li
Sabeer Thajudeen

9. Agenda
• About LinkedIn
• BI Challenges
• Administration Hacks
• Q&A

10. Challenges
• BIG DATA - 4 V’s : Velocity, Volume , Variety & Veracity
• Data Quality & SLA
• Navigation to reports/dashboards
• Multiple sources for answering critical questions

11. Metadata Driven Dashboards
by: Kevin Simon, Sr Data Warehouse Engineer
Microstrategy Meetup
December 4, 2013
LinkedIn Confidential ©2013 All Rights Reserved

12. FOR THE BUSINESS USERS:
How much would you be benefited if you
had the ability to effect real time change to
your critical dashboards, with little to no IT
involvement?
FOR THE ENGINEERS:
How much would you be benefited if your
users could make changes to their critical
dashboards, without needing to engage you
to make simple changes?

13. THE OLD WAY …
The fixed layout of this
dashboard makes it
difficult to manage.

15. HOW DID WE FIX THE PROBLEM?
With a Metadata driven design…
Real-time changes can effectively be made to
dashboards.
IT resources can be freed to work on other
projects.
Metrics can be introduced into your production
dashboards, prior to ETL development.

16. PROCESS FLOW AND TABLE DESIGN
Daily ETL
Aggregate
Process
Daily Custom
Metrics
Forecasting
Model (R)
Dashboard
View
Aggregate
DATE_SK
METRIC_SK
COUNTRY_ID
LOCALE_ID
METRIC_NAME
METRIC_VALUE
2013-11-01
100
500
1000
PAGE VIEWS
1000
2013-11-01
200
500
1000
SUBSCRIBERS
2000
2013-11-01
9000
550
1000
PV / UNIQUE MBRS
1500
Look up
METRIC_SK
METRIC_NAME
CAT_ID
CAT_ORDER
METRIC_ORDER
DISPL_FACTOR
100
PAGE VIEWS
100
10
10
200
SUBSCRIBERS (M)
100
10
20
1000000
9000
PV / UNIQUE MBR
200
20
10
1000

19. IN CONCLUSION…
A Metadata approach to dashboard
design has allowed for real time change
to critical dashboards.
More power and control given to users.
IT Resources can focus on other things.
Time to market greatly reduced.
Same message across multiple dashboards.

20. Next Play

21. Challenges
• BIG DATA - 4 V’s
• Data Quality & SLA
• Navigation to reports/dashboards
• Multiple sources for answering critical questions

22. Data Quality & SLA
by: Sabeer Thajudeen, Senior Software Engineer
Microstrategy Meetup
December 4, 2013
LinkedIn Confidential ©2013 All Rights Reserved

23. Dashboard Delivery
Daily ETL
Aggregate
Process
M1
Daily Custom
Metrics
M2
Forecasting
Model (R)
……
Mn
MSTR Workflow
Metric
Alert
WoW
No
Data
M1
Y
20%
Y
M2
Y
NA
Y
M3
N
NA
NA
Trigger Cube
Workflow
Scheduler
Data
Validation
(BTEQ)
Command Manager
- Trigger Event
(Alert Report)
Command Manager
Validator
Trigger
Internal
Trigger
External
ABORT
Command Manager
- Trigger Event
(Dashboard)

24. Challenges
• BIG DATA - 4 V’s
• Data Quality & SLA
• Navigation to reports/dashboards
• Multiple sources for answering critical questions

25. Navigation to reports/dashboards
by: Bhavani Akunuri, Senior Software Engineer
Microstrategy Meetup
December 4, 2013
LinkedIn Confidential ©2013 All Rights Reserved

26. Too Many Clicks

27. Resolution
• Identified top used reports
• Create short links
• Landing page re-design
Demo

28. Sample Code
Last Updated Timestamp
<web:ifBeanValue bean="frame" property="getDesignMode" value="0">
<web:then>
<div class="mstrMenuLastUpdatedCustom"> <web:displayGuiComponent
name="report_last_update" isContainer="true"/> </div>
</web:then>
<web:else>
<web:ifBeanValue bean="frame.rb" property="getXMLStatus" value="6">
<web:then>
<div class="mstrMenuLastUpdatedCustom"> <web:displayGuiComponent
name="report_last_update" isContainer="true"/> </div>
</web:then>
<web:else>
<web:displayGuiComponent name="report_design_area" isContainer="true"/>
</web:else>
</web:ifBeanValue>
</web:else>
</web:ifBeanValue>

29. Challenges
• BIG DATA - 4 V’s
• Data Quality & SLA
• Navigation to reports/dashboards
• Multiple sources for answering critical questions

30. Q: How are my key products performing?
Q: What caused the variance on KPI?
Q: Does it have a global effect?
Demo

31. Sample Code
Overlay Effect
Pop Up Code
<style>
.overlay_effect-ingraphs,
{
display:none;
position:fixed;
opacity:0.7;
height:50%;
width:50%;
z-index:100;
}
</style>
<script type="text/javascript">
$(document).ready( function() {
var main = $(this);
$("#ingraphs").click(function() {
loadPopupBox(".overlay_effect-ingraphs",".popupInGraphs");
});
function loadPopupBox(overlay,popup){
$(overlay).fadeIn("slow");
$(popup).fadeIn("slow");
}
});
</script>

32. Sample Code
Overlay Effect
Pop Up Code
<style>
.overlay_effect-ingraphs,
{
display:none;
position:fixed;
opacity:0.7;
height:50%;
width:50%;
z-index:100;
}
</style>
<script type="text/javascript">
$(document).ready( function() {
var main = $(this);
$("#ingraphs").click(function() {
loadPopupBox(".overlay_effect-ingraphs",".popupInGraphs");
});
function loadPopupBox(overlay,popup){
$(overlay).fadeIn("slow");
$(popup).fadeIn("slow");
}
});
</script>

33. Next Play

34. Agenda
• About LinkedIn
• BI Challenges
• Administration Hacks
• Q&A

35. Administration Hacks
by: Sabeer Thajudeen, Senior Software Engineer
Microstrategy Meetup
December 4, 2013
LinkedIn Confidential ©2013 All Rights Reserved

36. Swapping iServer names in User Bookmarks
Context:
▪ Disaster (DR) recovery is critical . MSTR infrastructure has been set up for DR
▪ Each time we test DR, VIP and base URL remains the same. However, iServer names change
Impact:
▪ Bookmarks (to access reports directly) include iServer names and will not work
Resolution:
▪ Use MSTR SDK to seamlessly swap incorrect iServer names in bookmarks (External Security
Module)
Link

37. Implementation Detail
▪ Create a Custom ESM (External Security Module) that extends ‘Abstract External
Security’
▪ Override the function ‘handlesAuthenticationRequest’
– 1) Read iServer names in incoming URL using regex. Matcher and
regex.Pattern libraries
▪ Pattern pat_type_1 = Pattern.compile("Server=.{4}-rpt..", 2);
▪ Pattern pat_type_2 = Pattern.compile("mstrWeb=-.{5}-rpt..", 2);
for(Matcher mat_type_2 = pat_type_2.matcher(requestURL); mat_type_2.find();)
{
//read iServer name in incoming URL
String iServerName = mat_type_2.group();
….
– 2) Replace if iServer name does not match expected server name
Example:
//Does iServerName in URL match with expected Server Name
if(! iServerName .equalsIgnoreCase((new StringBuilder("mstrWeb=-")).append(Location).append("*").append(Server1).toString()) && ! iServerName .equalsIgnoreCase((new StringBuilder("mstrWeb=")).append(Location).append("*-").append(Server2).toString()))
{
……
//Correct Server Name
requestURL = mat_type_2.replaceAll((newStringBuilder("mstrWeb=-")).append(Location).append("*").append(Server1).toString());
….}
}
– 3) Resubmit corrected URL - response.sendRedirect(requestURL);
▪ Update MicroStrategy Config Settings – Application Settings to use Custom ESM
(Web Customization Editor)

38. Monitoring MSTR via Web
Context:
▪ 24/7 Operations team to monitor and support infrastructure
▪ Handles not just MSTR but Teradata, Hadoop & ETL pipleline
▪ Prefer a web based system for monitoring rather than a desktop client
Solution
▪ Built a Web GUI to monitor critical aspects of the MicroStrategy infrastructure
– Delivery Status of all our critical dashboards
– Cube Refresh Status
– Job monitor for currently active jobs
– User Connection Monitor
MSTR Monitor
(Web Application)
InSights Monitor
JDBC
Statistics
Tables
SDK
MSTR
Application
(iServer)

39. Implementation Detail
▪ Dashboard Delivery Status and Cube Status
– IS_DOCUMENT_STATS and IS_CUBE_REP_STATS
▪ Instantiate WebObjectsFactory:
factory = WebObjectsFactory.getInstance();
▪ Force connection to each iServer
//Get cluster information
WebClusterAdmin wca = factory.getClusterAdmin();
WebClusters wcs = wca.getClusters();
for(int i=0;i<wcs.size();i++){
for(int j=0;j<wcs.get(i).size();j++){
//Set load balance factor to 0 to force cionnection to other iServer
wcs.get(i).get(j).setLoadBalanceFactor(0);
jobDetails+= getJobDetails(session,factory,httpReqId);
wcs.get(i).get(j).setLoadBalanceFactor(1);
}
}
▪ Inside getJobDetails:
//Retrieve job source. Loop through each job to get job info
//Use the same logic for the source WebMonitorTypeUserConnection to obtain session info
strSessionID = session.getSessionID();
source = (JobSource) factory.getMonitorSource(EnumWebMonitorType.WebMonitorTypeJob);
results = source.getJobs();
for (int i=0; i < results.getCount(); i++) {
Job singleJob = results.get(i);
job_id= singleJob.getJobID();
duration = singleJob.getDuration();….
}

40. Next Play

41. Credits
▪ Vincent Kwong – Program Manager
▪ Anirudh Gupta & Crowe Bonnie – Meet up Organizers
▪ LinkedIn AV team, Communications & Security
▪ Data Science & Biz-Ops
▪ ETL & Platform Teams
▪ NOC, DS Operations & DBA

42. Agenda
• About LinkedIn
• BI Challenges
• Administration Hacks
• Q&A