Design Specifications Document for the New Open Source Leadership Management. For applicability - please see The New Open Source Leadership Management Dashboard (LMD) Powerpoint Slides.
Unleash Your Potential - Namagunga Girls Coding Club
Leadership Management Dashboard Open Source Design Document
1. Leadership Management
Dashboard Version 4.1
(LMD 4.1)
System Design Specification
Prepared for
The National Institute of Food and Agriculture
By Barquin International
Last Updated March 30, 2011
2. LMD 4.1 System Design Specification
CONTENTS
1. Introduction ....................................................... 1
1.1. Background .................................................................................... 1
1.2. Document Purpose ........................................................................ 1
1.3. Assumptions and Constraints ........................................................ 1
2. Overview ........................................................... 2
2.1. Current Architecture....................................................................... 2
2.2. Architecture for Release 4.0 .......................................................... 3
2.2.1. Server Architecture Diagram ..................................................... 3
2.2.2. Server Communication Overview ............................................. 4
2.2.3. Database Schema Overview..................................................... 5
2.2.4. Function Summary .................................................................... 9
2.2.5. Access to LMD is via the NIFA Applications Portal ................ 10
2.2.6. LMD Component Data Flow Summary ................................... 11
2.2.4. LMD Component Data Flow Details ........................................ 12
3. Existing Functionality To Be Migrated ............. 16
3.1. Portfolios and NIFA Trends Tab Changes .................................. 16
3.2. Existing LMD Menu Structure ...................................................... 20
3.3. Menu Changes for LMD 4.0 ........................................................ 21
4. New Functionality ............................................ 28
4.1. LMD 4.0 Goals Tab...................................................................... 28
4.2. LMD 4.0 Portfolios Tab ................................................................ 31
4.3. LMD 4.0 Analyze Tab .................................................................. 36
4.3.1. The “Compare” Page .............................................................. 36
4.3.2. The “Filter” Page ..................................................................... 42
4.3.3. The “Rank” Page ..................................................................... 44
4.4. Drill to Detail Using the Project Count Measure .......................... 47
4.5. Saving and Accessing Saved Queries ........................................ 51
4.6. Sparkline Display and Drill to Detail ............................................ 55
4.7. Automatic Hierarchy Display in the Filter User Interface ............. 57
4.8. Version 4.1 Feature: Create Public Pages .................................. 62
4.9. Version 4.1 Feature: Aggregation Type....................................... 64
4.10. Version 4.1 Feature: Site Usage Tracking ................................. 65
4.11. Source Code Overview ................................................................ 66
4.12. JBoss Application Server Directory Structure ............................. 69
March 30, 2011
3. LMD 4.1 System Design Specification
Revision History
Date Version Description Custodian/Organization
Feb. 22, 2010 0.6 New goals for Version 4.0 Owen Dall
May 18, 2010 0.7 New goals for Version 4.0 Haroon Jaffer
Aug 17, 2010 0.8 Changed Menus for 4.0 Owen Dall
Aug 30, 2010 0.9 Documented recent enhancements Owen Dall
Aug 31,2010 0.9 Added page save and search features Owen Dall
Sep 1, 2010 0.9 Added sparkline section Owen Dall
Sep 2, 2010 1.0 Data model overview Owen Dall
Revised architecture diagrams Owen Dall
Documented table changes
Revised server port communication
Sep 8, 2010 1.01 diagram
Created new LMD component data flow
diagram.
Added section on Hierarchy display
Added JBoss Application Server Owen Dall
January 14,2011 1.02
Structure
March 30,2011 1.1 Added New 4.1.features Owen Dall
March 30, 2011
4. LMD 4.1 System Design Specification
1. Introduction
This document outlines the system design specification for Version 4.1 of the REEIS Leadership
Management Dashboard (LMD). This supersedes the 4.0 version.
1.1. Background
REEIS-LMD was initially developed based on the Oracle Portal Platform. In order to develop an Open
Source Framework, NIFA will migrate the As-Is REEIS-LMD Framework away from the Proprietary Oracle
development framework – onto the To-Be Open Source Framework.
1.2. Document Purpose
This System Design Specification Document defines the content and functionality that will migrated from
the As-Is Framework to Version 4.0 as well as additional functionality not currently available in LMD. This
Document also serves as a Communications Document to inform Stakeholders of the future look and feel
of REEIS-LMD. Since there already exists a detailed design document for LMD 3.4, that will not be
repeated here.
1.3. Assumptions and Constraints
This section presents all the known assumptions and constraints for the project.
Assumption #1 – The Open Source (To-Be Environment) will consist of the following Platform:
Red Hat Linux Operating System
Apache Web Server
JRuby
MySQL
InfoBright Plugin for MySQL
Mondrian
Rails
Hobo
Assumption #2 – Only the content specified for Version 4.0 will be migrated to the Open Source
environment. The remaining LMD features will continue to reside on the existing Oracle environment.
March 30, 2011 Page 1
5. LMD 4.1 System Design Specification
2. Overview
This section provides a brief description of the steps that are necessary to complete the transition from
the current As-Is REEIS-LMD environment to the To-Be REEIS-LMD environment and major tasks
involved in this transition. The transition will be implemented in Phases.
2.1. Current Architecture
March 30, 2011 Page 2
6. LMD 4.1 System Design Specification
2.2. Architecture for Release 4.0
2.2.1. Server Architecture Diagram
The following diagram outlines the proposed architecture to support LMD Version 4.0:
March 30, 2011 Page 3
7. LMD 4.1 System Design Specification
2.2.2. Server Communication Overview
Apache http://portal.nifa.usda.gov Open to the web. Hides the App
Proxy Server and DB servers from outside
access.
JBoss App prod-lmd-app.csrees.usda.gov Runs the application code.
Server Hidden behind the firewall.
MySQL DB prod-nifa.bd.csrees.usda.gov Runs the MySQL /InfoBright
Server database server. Hidden behind
the firewall.
AD Server NIFA Active Directory (authentication) server Used by the App server to
authenticate NIFA users. Behind
the firewall.
REEIS/LMD reeispw.nifa.usda.gov Hosts Oracle Portal, REEIS Web
Oracle App site, and provides authentication
Server to REEIS LMD 3.4
March 30, 2011 Page 4
8. LMD 4.1 System Design Specification
2.2.3. Database Schema Overview
There are two types of tables used in the Open Source portion of LMD 4.0:
1. Star Schema. The InfoBright database engine (“BRIGHTHOUSE”) plug-in for MySQL (read
only) that contain the measures and dimensions accessed via the Mondrian Dimensional
eXpression (MDX) interface from Ruby. The following diagram shows the Project Financials
Star Schema structure:
LMD_PROJECTS_FINANCIAL_FACTS Master fact table
LMD_PROJECTS Project Dimension
LMD_GRANTEES Grantee Dimension
LMD_FUNDING_SOURCES Funding Source Dimension
LMD_YEARS Year Dimension
LMD_KNOWLEDGE_AREAS Knowledge Area Dimension
LMD_SUBJECTS Subject Dimension – CRIS Subject of Investigation
LMD_SCIENCES Science Dimension – CRIS Field of Science
March 30, 2011 Page 5
9. LMD 4.1 System Design Specification
2. Support Tables. MySQL ISAM database engine (“MyISAM”) tables for the site infrastructure
and saved pages (Ruby/Rails/Hobo):
site_configs Saves default pages for all tabs and other site
configuration information.
annotations <Reserved for future use?
filter_pages Page repository for all user-defined filters
comparison_pages Page repository for all user-defined
comparisons
rank_pages Page repository for all user-defined rankings
goal_summary_pages Page repository for all user-defined goal
summaries
goal_trend_pages Page repository for all user-defined goal
trends
portfolio_summary_pages Page repository for all user-defined portfolio
summaries.
portfolio_trend_pages Page repository for all user-defined portfolio
trends
users <default Hobo table>
schema_migrations <default Hobo/Rails table>
March 30, 2011 Page 6
10. LMD 4.1 System Design Specification
March 30, 2011 Page 7
11. LMD 4.1 System Design Specification
The following figure shows all tables with the database engine type for each and the latest record
count for each table:
March 30, 2011 Page 8
12. LMD 4.1 System Design Specification
2.2.4. Function Summary
This table outlines, by function, the infrastructure components currently used and the
replacements for LMD 4.0.
FUNCTION CURRENT LMD 4.0
Virtualization N/A VMware Enterprise
Operating System Windows 2003 Linux - Red Hat Enterprise 64bit
Application Server Oracle IAS 10G JBoss/Tomcat
Database Server Oracle 10G Enterprise MySQL
OLAP Server N/A InfoBright Community Edition
Authentication Oracle OID, LDAP CAS/LDAP
Content Management (CMS) Oracle Portal TBD
Portal/Page Flow Oracle Portal Ruby/Rails/Hobo
Site Search Concept Search Google Search Appliance
Dynamic Pages Oracle Portal Ruby/Rails/Hobo
Pixel-Perfect Reports Crystal Reports JasperSoft iReports
Cross-Tabular Reports Custom with PL/SQL Custom with Hobo/Rails/Ruby
Charts and Graphs Oracle Discoverer Fusion Charts/Flot/Other
Heat Maps N/A Google Web Toolkit with Overlays
ETL Data Junction/Oracle PL/SQL Talend ETL
Personalization (Settings) Barquin Engine Barquin Engine
March 30, 2011 Page 9
13. LMD 4.1 System Design Specification
2.2.5. Access to LMD is via the NIFA Applications Portal
Access to LMD will require authentication via the NIFA Applications Portal at http://portal.nifa.usda.gov.
(Details about the design of the NIFA Applications Portal and the user authentication and authorization
process are available in a separate document.)
Once a user logs in via the NIFA Applications Portal, and if authorized to use LMD, a link to access LMD
will be displayed after login:
March 30, 2011 Page 10
14. LMD 4.1 System Design Specification
2.2.6. LMD Component Data Flow Summary
March 30, 2011 Page 11
15. LMD 4.1 System Design Specification
2.2.4. LMD Component Data Flow Details
Server Initialization
1. The JBoss application server starts up with the /etc/init.d/jboss initialization
script.
2. JBoss loads an instance of the LMD4 Rails application and sets it up to listen on /lmd4.
This instance serves multiple concurrent requests through use of multithreading.
3. The Rails application instance opens its ActiveRecord connection to the
InfoBright database (MySQL plug-in).
First Query after the Server Initialization
1. This is the first time results are shown after the server is up.
2. The Mondrian Domain Specific Language (MDSL) wrapper library loads the Mondrian
Java libraries into memory. (MDSL is written in Ruby.)
3. The MDSL library checks to see if it is connected to InfoBright.
4. Since this is the first try, InfoBright has not been connected.
5. MDSL builds a Mondrian ROLAP connection object (RolapConnection) that holds a
connection to the database, ready to execute MDX queries.
Request Cycle on the Filter Page (other pages are similar)
When the user selects a page tab:
1. The Request hits http://-- - - -/filter_pages/new.
2. The Apache HTTP server forwards the request to the JBoss applications server.
3. JBoss passes the request to the Rails application.
4. The Rails app renders a default Filter page.
When the user selects a filter on a page:
1. The Filter, sublevel and the chosen value are stored in a JavaScript array called
active_filters.
2. The selected value is removed from the dynamic dropdown box so it cannot be chosen
twice.
March 30, 2011 Page 12
16. LMD 4.1 System Design Specification
When the user clicks the delete icon next to a chosen filter to remove it:
1. The Filter, sublevel and value are removed from the active_filters JavaScript array.
2. The value is put back in the dynamic dropdown box.
When the user clicks the “Show Results” button:
1. The chosen dimensions, measures and the active_filters JavaScript array are bundled up
into a JSON object.
2. The JSON object is sent via AJAX to the application, which routes the request to the
FilterPagesController's view_only action.
3. The view_only action passes the parameters to the model, which sets up an unsaved
Filter Page instance (called filter_page.)
4. The view_only action then queries the filter_page object to get the results of the query.
5. The filter_page uses the parameters passed in from the UI to construct an MDX query
using the MDSL ruby library's convenient interface to the MDX language.
6. Once constructed, the MDX query string is given to the MDSL Schema class to execute.
7. The Schema class checks to see if the Mondrian Java libraries have been loaded and
loads them up if they have not been loaded.
8. Once loaded, the Schema class uses the Mondrian RolapConnection object to open a
connection to InfoBright.
9. The MDX query string - built in an earlier step – is then handed to this RolapConnection
object which executes the query.
10. The result is returned as an instance of the java class 'RolapResult'.
11. This result is parsed and wrapped into an instance of class MDSL::ParsedResult which
then provides a clean interface to the returned data.
12. Once the result is parsed into MDSL::ParsedResult, it is returned to the controller.
13. The controller plugs the result into a Rails partial [_hierarchy_table.html.erb] which
turns it into the result table.
14. It also passes the result into a different Rails partial [_fusionchart_single_series.rxml]
March 30, 2011 Page 13
17. LMD 4.1 System Design Specification
which formats the results for building the FusionChart.
15. The third part of the response is the original MDX query string that will be displayed
when the 'MDX' icon is clicked. This icon is located immediately above the displayed
table of results, next to the CSV export icon. (This information can be used for
debugging purposes.)
16. These three parts are converted into a JSON string and send as the response to the
browser:
a. Results Table
b. FusionChart XML
c. and the MDX query string
17. The browser receives the response and decodes the JSON string.
18. The table HTML is plugged into the page and displayed.
19. The XML is used to build a FusionChart and an associated FusionChartExporter is also
created.
20. The MDX query string is plugged into an invisible HTML <div> that gets transformed
into a modal box and shown when the MDX button is clicked.
When the user clicks the “Save This Page” button:
1. A JavaScript prompt asks the user to name the page before saving it.
2. The user-provided title, along with the same parameters passed on “Show Results” are
sent to the Rails controller.
3. The controller builds a filter_page object with the parameters and saves it to the
database.
4. The user is redirected to the saved page.
When the user clicks the “CSV” button:
1. The same process holds as for the “Show Results” button, but instead of returning a
JSON response, the results are sent to the browser formatted as CSV for download.
When the user clicks the “Save” button on a previously saved page:
1. The contents of the form and the active_filters are sent to the controller.
2. The controller updates the page's details with the form parameters.
March 30, 2011 Page 14
18. LMD 4.1 System Design Specification
3. The controller redirects the browser back to the same page with a message displayed that
informs the user of the successful update.
When the user clicks the “Save as New Page” button on a previously saved page:
1. A JavaScript prompt asks the user for a unique title for the new page.
2. The contents of the form, the active_filters and the provided title are sent to the
controller.
3. The controller builds a new filter_page and saves it with the passed parameters and
under the given title.
4. The browser is redirected to the freshly created page.
March 30, 2011 Page 15
19. LMD 4.1 System Design Specification
3. Existing Functionality To Be Migrated
In this section of the document we will outline the existing content and functionality that will be migrated to
LMD 4.0. In the next section we will outline new functionality and content that does not currently exist. In
the “As-Is” LMD.
3.1. Portfolios and NIFA Trends Tab Changes
CURRENT LMD 4.0
Portfolios Tab Portfolios Tab
- Knowledge Area Page - Summary Page
- Trends Page
- NIFA Funding Sources Page
o Expend/Collapse Option
- All Funding Sources Page
o New Charts
All with sections including the past 5 years of data, with
row and column totals for the following Measures: All with sections including the past 5 years of data, with
row and column totals for the following Measures:
o Consolidated Dollars (“Funding”)
o Scientist FTEs o Consolidated Dollars
o Project count o Scientist FTEs
o Project Count
- New:
o Expenditure Dollars
o Award Dollars
NIFA Trends
- Portfolio Trends
- By Funding Mechanism - Goal Trends
- Trends in the Filter Module
- Goals & Objectives - Trends in the Compare Module
March 30, 2011 Page 16
20. LMD 4.1 System Design Specification
Figure 1: "Portfolios" Tab Contents
March 30, 2011 Page 17
21. LMD 4.1 System Design Specification
Figure 2: "NIFA Trends" tab contents – Tab 1 of 2 – Funding Mechanism
March 30, 2011 Page 18
22. LMD 4.1 System Design Specification
Figure 3: "NIFA Trends" tab contents - Tab 2 of 2 - Goal
March 30, 2011 Page 19
23. LMD 4.1 System Design Specification
3.2. Existing LMD Menu Structure
Existing Menu Structure in LMD 3.4
The existing top-level menu structure for LMD 3.4 is as follows:
MENU ITEM DESCRIPTION
My Snapshots Yearly snapshots of data based on values selected in Settings.
My Projects Personalized project alerts, digests, and summaries.
NIFA Trends Trend graphs by NIFA formula funding categories and Goals and Objectives.
Trends in spending, FTEs, and project counts organized by NIFA Portfolios and
Portfolios
Knowledge Areas with Portfolios.
All projects in the system broken down by sponsoring agency, including NIFA,
Agency Projects
ARS, FS, ERS, or State Partners.
Workload Workload of NPLs by Unit and Section and type…
Detailed snapshots regarding Plans and Annual Reports for selected Formula
funds, with views from Strategic Goals perspective, Portfolio Perspective, Funding
AREERA POW
Perspective, Geographical perspective, Institution perspective. Also search
outcomes by keyword, portfolio, goal, or Knowledge Area.
Personalize views of Projects and Snapshots by the following dimensions:
Settings Program Area, Program, FDC, Knowledge Area, Subject of Investigation, Field of
Science, and Location.
Definitions Web view of important terminology used in LMD
Contact Us Page to send a comment or request
Logout Log out of LMD
March 30, 2011 Page 20
24. LMD 4.1 System Design Specification
3.3. Menu Changes for LMD 4.0
Proposed Top Level Menu Structure for LMD 4.0
For LMD 4.0 we will remove 1) “NIFA Trends” and 2) “My Snapshots” as a separate items, as each
content area will house its own “Trends” and “Summaries” tab.
ITEM LOCATION DESCRIPTION
Activities Oracle Portal All Post award activity tracking (e.g. Projects) such as “My Projects”,
“Agency Projects”
My Snapshots Oracle Portal Knowledge Area, Subject of Investigation, Field of Science
State Snapshots Oracle Portal State views by Funding Mechanism, Organization, and Top 10 KAs and
SOIs.
Awards Oracle Portals Award information by FDC (and rollups) and Program (and rollups), with
drill-down to individual projects.
AREERA POW Oracle Portal AREERA Plans and Annual Reports for selected Formula Grants.
Workload Oracle Portal Workload of NPLs by Unit and Section and type…
Goals Hobo/Rails Summaries and Trends in spending, FTEs, and project counts organized by
NIFA Goals, Objectives, and Knowledge Areas with Portfolios.
Portfolios Hobo/Rails Trends in spending, FTEs, and project counts organized by NIFA Portfolios
and Knowledge Areas with Portfolios.
Analyze Hobo/Rails Advanced comparison and filtering options for all data
Definitions Hobo/Rails Web view of important terminology used in LMD
Contact Us Oracle Portal Page to send a comment or request
March 30, 2011 Page 21
25. LMD 4.1 System Design Specification
Separate Images/Hyperlinks above the main menu for these options:
Settings Oracle Portal
Back to Portal Both
Logout Both
The following screen shot shows the equivalent menu structure in the Open Source portion of LMD:
March 30, 2011 Page 22
26. LMD 4.1 System Design Specification
The complete menu hierarchy for LMD 4.0 is documented in the following table
Menu Level 1 Menu Level 2 Menu Level 3 New Open Source
Activities My Projects Alerts
Digest
Summary
Multistate
Agency Projects Alerts
Digest
Summary
Multistate
My Snapshots Knowledge Area
Subject
Field of Science
State Snapshots Funding Mechanism Hierarchy
Organization Hierarchy
Top 10 Knowledge Areas
Top 10 Subjects
Awards Program Area
Programs
FDCs
AREERA POW National Summary
Strategic Goal Perspective
Geographic Perspective
Portfolio Perspective
Knowledge Area Perspective
Funding Source Perspective
Institution Perspective
Outcomes Search by Goal
Outcomes Search by Portfolio
Outcomes Search by
Workload Knowledge Area
Goals Summary
Trends
Portfolios Summary
Trends
Analyze Filter
Compare
Rank
Definitions
Contact Us
Separate Buttons/links for “Settings’, “Back to Portal”, and “Logout”. ”Settings” results in the display of
the following eight tabs:
Settings NPL Proxy
Program Area
Program
FDC
Knowledge Area
Subject
Field of Science
Location (State)
March 30, 2011 Page 23
27. LMD 4.1 System Design Specification
Activities:
My Snapshots:
State Snapshots:
Awards:
March 30, 2011 Page 24
28. LMD 4.1 System Design Specification
AREERA POW (menus as links):
Workload (no sub-menus):
March 30, 2011 Page 25
29. LMD 4.1 System Design Specification
Goals:
Portfolios:
Analyze:
Definitions:
March 30, 2011 Page 26
30. LMD 4.1 System Design Specification
Contact Us:
Settings:
“Logout” results in the display of the NIFA Applications Portal logout screen:
March 30, 2011 Page 27
31. LMD 4.1 System Design Specification
4. New Functionality
4.1. LMD 4.0 Goals Tab
The Goals tab has two tabs, one for a snapshot “Summary” and the other for “Trends.”
Note: In this section we will focus on the Trends tab functionality, which also applies to Portfolios. In the
Portfolios section we will document the “Summary” function, which is also available for Goals.
The LMD 4.0 "Goals Trends" Page – Part 1 of 2
March 30, 2011 Page 28
32. LMD 4.1 System Design Specification
Note: The “Export” command exports data which is expanded to the lowest level, i.e. Goals -> Objectives
-> Knowledge Areas. Clicking the “Show Results” button resets the report to show data at the highest
level
Figure 4: LMD 4.0 "Goals Trends" Page – Part 2 of 2
Note: “Click to export chart as JPEG” is a two step process.
Step 1: Click the button to export cart as JPEG (once capturing is complete Save will button will become
available just beneath Export button.
Step 2: Click Save to save the file with .jpg extension to your local file system.
March 30, 2011 Page 29
33. LMD 4.1 System Design Specification
Figure 5: The user can switch from Column to Line views
March 30, 2011 Page 30
34. LMD 4.1 System Design Specification
4.2. LMD 4.0 Portfolios Tab
The Portfolios tab infrastructure (Summary and Trends) is identical to that of the Goals Tab, but uses a
different rollup Hierarchy. In this section we will focus on the Summary tab.
Figure 6: LMD 4.0 "Portfolio Summary" Page – Part 1 of 4
Note: the “Delete page” option will only be available to LMD Application Administrators.
March 30, 2011 Page 31
35. LMD 4.1 System Design Specification
Figure 7: "Portfolio Summary" Page - Part 2 of 4
March 30, 2011 Page 32
36. LMD 4.1 System Design Specification
March 30, 2011 Page 33
37. LMD 4.1 System Design Specification
Figure 8: Portfolio Summary Page - Part 3 of 4
Figure 9: “Portfolio Summary” Page – Part 4 of 4
March 30, 2011 Page 34
38. LMD 4.1 System Design Specification
March 30, 2011 Page 35
39. LMD 4.1 System Design Specification
4.3. LMD 4.0 Analyze Tab
The Analyze tab is completely new in LMD 4.0. There are three sub-tabs within Analyze:
Compare
Filter
Rank
4.3.1. The “Compare” Page
The Compare Page is designed to provide the ability to compare any two items on any dimension
available.
Years. By default the years available within the data mart are displayed next to the “Years” prompt. The
user can select a start and end year to include in the comparison:
March 30, 2011 Page 36
40. LMD 4.1 System Design Specification
Measure Type. The user selects one measure from the list available as the metric for comparison.
Currently the measures available are:
Compare. The user chooses a dimension for the comparison. All of the available dimensions and rollups
of dimensions are available:
March 30, 2011 Page 37
41. LMD 4.1 System Design Specification
Filter. Any number of optional filters can be used for the comparison. The same list of dimensions is
available for filtering:
March 30, 2011 Page 38
42. LMD 4.1 System Design Specification
Results. Clicking on the “Show Results” button will execute the query and return a table, sparklines, and
calculations for “Absolute” and “Percentage” difference calculations.
The default graphic is a 3-D Column (bar) chart representation of the two items in the comparison, as well
as the absolute difference between them. The user can select and rotate this chart as needed for viewing
details.
March 30, 2011 Page 39
43. LMD 4.1 System Design Specification
The 3-D Chart can be rotating using the mouse see the inner bars:
Selecting “Lines” from the dropdown list labeled “Visualize information as “ brings up an alternative
display:
March 30, 2011 Page 40
44. LMD 4.1 System Design Specification
As always with LMD charts, the user can export a chart as a JPEG graphic (.jpg) file that can easily be
imported into any Word, PowerPoint, or Excel document.
March 30, 2011 Page 41
45. LMD 4.1 System Design Specification
4.3.2. The “Filter” Page
The Filter page provides the user the ability to filter/focus on a subset of data in the data mart by
choosing:
1. Measure Type
2. Dimension for Rows
3. A Dimension for Columns
4. Filters
As always, “Show Results” executes the query and displays a table of results and a chart below the
definition portion of the page:
March 30, 2011 Page 42
46. LMD 4.1 System Design Specification
March 30, 2011 Page 43
47. LMD 4.1 System Design Specification
4.3.3. The “Rank” Page
The “Rank” page is designed to provide a quick method of comparing items in any dimension by the
measure of choice. There are only three options for this page:
1. Measure Type
2. Dimension for Rows
3. Filters
For example, ranking states by how many Award Dollars were received by the sponsoring agency NIFA
would be specified as follows:
March 30, 2011 Page 44
48. LMD 4.1 System Design Specification
The results:
March 30, 2011 Page 45
49. LMD 4.1 System Design Specification
And a chart for visualizing the ranking:
March 30, 2011 Page 46
50. LMD 4.1 System Design Specification
4.4. Drill to Detail Using the Project Count Measure
Drill to Detail
When the user picks “Project Count” as the measure of interest in any of the Goals, Portfolios, or Analyze
pages, values in the resulting cells of any page become hot links to drill to a project listing page:
The Project List page is then displayed:
March 30, 2011 Page 47
51. LMD 4.1 System Design Specification
Clicking on an “Accession Number” will show the related Project Detail page:
Project List Options
There are three options available to the user on this page:
1. Columns
2. Export
3. Rows per Page
March 30, 2011 Page 48
52. LMD 4.1 System Design Specification
Columns
Selecting this option will bring up a checkbox list of all available columns to display:
Select the desired fields to display and click on the “Choose” button:
Export
This option provides the ability to export all of the information currently active in the Project List to a
comma separated value (CSV) format that can be loaded into Excel or another software package for
analysis.
Rows per Page
Allows the user to set the preference for how many rows of data should be set per page.
March 30, 2011 Page 49
53. LMD 4.1 System Design Specification
The first row of any export will contain the column names including in the export:
March 30, 2011 Page 50
54. LMD 4.1 System Design Specification
4.5. Saving and Accessing Saved Queries
A recent addition to the functionality of LMD 4.0 is the ability to create and name new pages using user-
defined comparisons, filters or rankings. In version 4.0, all saved pages will be available to all LMD
users. (It is planned to allow saving private pages in version 4.1).
In the example below, states will be ranked by the Award Dollars received from NIA for the Food Safety
Portfolio. Since there is no filter by year, all years of data (2002-2008) included in the data mart will be
used in the calculations:
The results:
March 30, 2011 Page 51
55. LMD 4.1 System Design Specification
Clicking on “Save This Page” will bring up a window for entering a title, description, and keywords for this
page to be saved:
After the “Save” button is clicked, the page is refreshed to look as follows:
March 30, 2011 Page 52
56. LMD 4.1 System Design Specification
Search Saved Pages. To access previously saved pages, click on the “Search saved pages” button:
In this case you can see that there have been three saved “Rank” pages
As the repository of saved pages grows, the search box becomes useful:
It uses a real-time type-ahead matching feature that does not require a “submit” action.
March 30, 2011 Page 53
57. LMD 4.1 System Design Specification
March 30, 2011 Page 54
58. LMD 4.1 System Design Specification
4.6. Sparkline Display and Drill to Detail
From Wikipedia:
“The term sparkline was proposed by Edward Tufte for "small, high resolution graphics embedded in a
context of words, numbers, images". Tufte describes sparklines as "data-intense, design-simple, word-
sized graphics"
Whereas the typical charts designed to show as much data as possible, and is set off from the flow of text
(as in the box below), sparklines are intended to be succinct, memorable, and located where they are
discussed.”
In LMD, the sparklines are placed at the end of the row of numbers, to make it easier to see trends at a
glance:
Looking at the sparklines in the chart above, we can see immediately which portfolios have had an
increase in project counts over time.
For example, Portfolio 4, “Education” has increased from 201 to 633 from 2002-2008.
Clicking on that sparkline will drill to a new table and chart that includes only that row of data:
March 30, 2011 Page 55
59. LMD 4.1 System Design Specification
This page has a more accurate line chart graph which can be switched to a bar chart view:
March 30, 2011 Page 56
60. LMD 4.1 System Design Specification
4.7. Automatic Hierarchy Display in the Filter User Interface
The LMD 4.0 user-interface is designed to automatically use the data model declaration of dimension
hierarchies (rollups) anywhere the user is prompted to provide a value for a dimension.
For example, the Knowledge Area dimension is represented in the table LMD_KNOWLEDGE_AREAS:
+
Knowledge Area has three rollups (hierarchies) that are used within NIFA, but currently we are only using
2 and 3. The first is the three level hierarchy that rolls up to TOPIC:
March 30, 2011 Page 57
61. LMD 4.1 System Design Specification
The second is the two-level rollup to PORTFOLIO:
The following Ruby code defines the Portfolio hierarchy:
The third is the three-level rollup to GOAL
March 30, 2011 Page 58
62. LMD 4.1 System Design Specification
And the relationships in the Knowledge Area model:
The definition above provides the following capability:
Choose “Knowledge Area” from the Filter list:
March 30, 2011 Page 59
63. LMD 4.1 System Design Specification
LMD next prompts you to select from the list of rollup levels available:
Then provides the list of values for the rollup level:
March 30, 2011 Page 60
64. LMD 4.1 System Design Specification
March 30, 2011 Page 61
65. LMD 4.1 System Design Specification
4.8. Version 4.1 Feature: Create Public Pages
This option allows the creator of a page (saved query) to flag to be shared. This means that the
URL generated for the page + the ending “/public” can be posted anywhere on the internet for
access.
For example, a standard LMD page view appears as follows:
Next the public view accessible via the following URL:
http://test-portal.nifa.usda.gov/lmd4/portfolios/summaries/11-public-page-test-portfolios-summary-for-2009/public
March 30, 2011 Page 62
66. LMD 4.1 System Design Specification
Functionality Summary
The “Share with public” link allows the creator of a saved page to publish a human-
friendly link that can be viewed anywhere.
This option can be turned off and on by the creator or the LMD Application
Administrator.
This public URL renders a page w/o the current top-level menu options, but keeps the
USDA and REEIS logos and links.
The “Create New Page”, “Show Results”, “Save”, and “Save as New Page” buttons are
removed as they are no longer relevant.
The user can still drill-to the project listing and project detail pages
March 30, 2011 Page 63
67. LMD 4.1 System Design Specification
4.9. Version 4.1 Feature: Aggregation Type
A user can now choose either “Sum” or “Mean” as the aggregation type for each page. This
allows comparisons and rankings not possible before. For example, ranking states by the mean
(average) reported expenditures for formula grants for 2009:
March 30, 2011 Page 64
68. LMD 4.1 System Design Specification
4.10. Version 4.1 Feature: Site Usage Tracking
A MySQL table called “usage_records” has been added that will track usage of each page and
action within a page. The following information is being collected:
Request URL (Full human-friendly URL used to access the page)
Page ID (the page ID in MySQL)
Page Type (Analyze, Filter, Rank, etc.)
Action Name (If internal, “Show Results”, “Create New Page”, etc.
User ID (for registered users)
Type of Controller (Public or Internal)
Referrer (URL of site that has the link posted, if any)
Default Page (Is the page hit a default page?)
Client IP (IP Address of the client browser
March 30, 2011 Page 65
69. LMD 4.1 System Design Specification
4.11. Source Code Overview
All of the source code for LMD 4.1 is contained within the LMD folder as outlined below. This code is
hosted at github.com (branch=”nifa-master”):
Also downloaded daily on the NIFA code repository at:
J:applicationscode_repositorydailiy_git_downloads{date}lmd
For example;
J:applicationscode_repositorydailiy_git_downloads2010-09-10lmd
And looks like the following:
March 30, 2011 Page 66
70. LMD 4.1 System Design Specification
The LMD application folder structure follows the normal Rails/Hobo structure for models, controllers, and
views, but also contains a new folder for the Mondrian Domain Specific Language (MDSL) created to
easily define data marts and hierarchy levels and integrate with the Mondrian Dimensional eXpression
language (MDX) that is used to communicate with many high-performance column store database
engines like InfoBright.
March 30, 2011 Page 67
71. LMD 4.1 System Design Specification
March 30, 2011 Page 68
72. LMD 4.1 System Design Specification
4.12. JBoss Application Server Directory Structure
/home/jboss The LMD4 application and supporting software are installed under this directory and it’s
subdirectories:
/certs Self-signed SSL certificate and SSL trust-store
/jruby JRuby installation
/lib/ruby This directory is copied over from dev to test (then from test to the rest) when required Ruby gem
upgrades and installations have occurred.
/java Java JDK installed here
/tomcat Home directory for the Tomcat Application Server
/webapps LMD Proxy Server (separate CAS proxy for single process application configuration)
/conf Configuration files for the Tomcat Application Server
/railsapps A directory for staging application source code before deployment
/lmd-dashboards/LMD Ruby on Rails code for the LMD4 app is first copied to this directory before being bundled into a
.war file for deployment to the JBoss Application Server
/lmd_proxy_server Source code for the LMD Proxy Server is staged here before deployment to the Tomcat
Application Server
/jboss Home directory for the JBoss Application Server
/server/default Lmd4.war is copied to this directory during deployment to JBoss AS
/conf JBoss Application Server configuration files
/deploy/jbossweb.sar More JBoss Application Server configuration files reside here
March 30, 2011 Page 69