This document discusses various customization tips and techniques for administering MicroStrategy Narrowcast Server that Bluegreen Corp. implemented. It describes problems they faced such as slow report execution and subscriptions not being linked to reports. It then details the solutions implemented, such as adding owner information to subscriptions, using dynamic reply-to addresses, and integrating Narrowcast data into their Enterprise Manager for exception reporting. Custom tools were also created to trigger subscriptions and analyze logs. The key lessons are to proactively monitor systems and address issues before they occur.
[2024]Digital Global Overview Report 2024 Meltwater.pdf
World 2008 - Administering Narrowcast Server
1. Customization Tips and Techniques for
Administering MicroStrategy Narrowcast Server
Bryan Brandow
Sr. MicroStrategy Developer
Bluegreen Corp.
bryan.brandow@bluegreencorp.com
January 16th, 2008
2. About Bluegreen
History
Founded in 1966 in Massachusetts, the company’s core business was
buying, improving, selling and financing recreational land in more than
32 states. In 1986 the company was listed on the New York Stock
Exchange and in 1991 it was relocated to Boca Raton, Fla.
Overview
Bluegreen Corporation (NYSE:BXG) is a leading provider of Colorful Places to Live and
Play®. Headquartered in Boca Raton, Fla., the company has over 6,000 associates
working in its Bluegreen ResortsSM and Bluegreen Communities® divisions.
Bluegreen Resorts markets and sells a flexible, real estate-based vacation ownership
plan with access to over 40 resorts and an exchange network of over 3,700 resorts and
other leisure products such as cruises and hotel stays. In 2006 Bluegreen Resorts
delivered nearly 300,000 unique vacation experiences to over 170,000 owners.
3. Where We Came From
• Static web based reports executing from copies of the transactional
tables would take 15-30mins and upwards of 2-3hrs for yearly reports.
• Delays in the queue would start in the morning and on busy report days,
a request could take hours to be fulfilled.
• In order to be prepared for important Monday meetings, users had to
come into the office earlier and earlier to beat the morning report rush.
• Users had to print large binders of reports and prepare them for
daily and weekly usage by Site Directors.
• No real data cleansing processes.
4. New System Requirements
Our users needed a more scalable and flexible tool, one that could support
our growing company’s report and analysis demands.
Old New
• Slow execution (minutes to hours) • Fast execution (seconds, regardless of load)
• Static report. • More flexible in prompts and drill downs.
• Scheduling
5. Narrowcast at Bluegreen
At Bluegreen, we exclusively use the Web Deliveries Integration as all
subscriptions are created from pre-existing reports that users run or create
themselves in MicroStrategy Web. This provides users with a single
interface to execute reports and manage their subscriptions.
Web Deliveries is a relatively new feature, debuting in 8.0. During our
implementation and roll out, we encountered some challenges. This
presentation takes you on a tour of some of the things we faced and our
solutions to overcome them.
6. Where Did This Come From?
Problem:
Users would receive subscriptions and have questions about them
but not know who to ask. They would ask IT, but we didn’t have a
way to track an email back to a subscription. Users were confused
and IT was frustrated!
Solution:
Adding Information
Objects for the owner
of the subscription and
the schedule that it is
associated with will help
in the debugging
process and let the
users know who to
contact if they have
questions.
7. Smart Reply To’s
Continued:
We also modified the emails sent from Narrowcast to use a dynamic
Reply To. The users would see the normal “Business Intelligence”
address as the sender, but if they clicked “Reply”, it would magically
change to the owner of that subscription.
• Email is received from the • Clicking reply replaces the name
standard company distribution box. with the user who created the
subscription.
8. Exception Reporting
Problem:
Some users have created exception
based reports to email them when
certain criteria are met. However,
an email is sent regardless of report
content and the user is given a daily
“No Data Returned” email.
Solution:
Edit the Information Object for
the report data and either change
the message to something nicer or
don’t send an email at all.
9. Timeouts!
Problem:
As our subscription base got higher, so did the load on our server.
Some services started to struggle to finish and timeout.
10. Quick Narrowcast Lesson
All of the information needed to
Service process the subscription (Who, What,
When).
Individual instance that a Service is
executed.
Subset of subscriptions broken out
Segments Segments
for each execution for use in load
balancing between multiple Execution
Engines.
11. Timeouts!
Problem:
As our subscription base got higher, so did the load on our server.
Some services started to struggle to finish and timeout.
Solution:
Proper usage of the Segment Size, Execution
Timeout and Work Allocation settings helped
improve system stability and performance.
12. Administrative Notifications
Be the First to Know!
• Here, you can customize your
notifications. Just put in your email
address, select the section that sounds
like what you’re after, and click the
Governing and Filtering button.
• Select either all notifications or specific
ones. Careful though, sometimes errors
that are logged aren’t necessarily
important ones! Some are simply optional
pieces of a subscription that a user chose
to leave blank.
13. What Really Happened …
Problem:
Users would claim that they didn’t receive a
subscription, or there was a problem with the report.
How are we to know what really happened?
Solution:
At this point in the project, we found that we were
mostly in the dark about what was happening. We
employed several different solutions to get a better
grasp on what was happening …
14. Saved Mail – A Life Saver!
Saved Mail
This setting on the Information
Transmitter will save a copy of every
email that it sends. This can be a life
saver for debugging what’s going on. Just
remember to periodically clean it out
because it gets very large!
• This is also a great way to do debugging in Production. If there is a service
giving you problems, but you don’t want to spam the users, change this setting
to “Save to File” and the users won’t receive any emails while you test!
15. Spy on the Inbox
•Setup a dedicated
email box for BI
reports to be sent
from, and add it to your
outlook so that you can
see its return
messages.
• You’ll be able to see what
happens to emails AFTER
they’ve left the hands of
Narrowcast. For example, if
someone’s email box is full or
their account is misspelled or
non-existent.
16. Log Files …
• Log files hold the keys to every
mystery. Be sure to enable additional
pertinent information so that it is
available when needed.
• Be sure to edit your log governing
settings! By default, they are 5mb and 10
files, that’s not enough!
Oh, there’s the problem …
17. Narrowcast Log Analyzer
• NCLog is a Visual Basic.NET
application written to help parse and
analyze Narrowcast log files.
• Instead of the normal
log file being stored
chronologically, NCLog
allows you to browse the
file logically.
• Predefined user alerts
are flagged with a red
bulb and bubbled up to
the top, indicating that a
particular service
contained an alert. You
can now drill into a service
and see only information
related to the service you
are interested in.
19. Narrowcast Log Analyzer – Other Toys
• Copy the full log file line to clipboard.
• Export this branch to file.
• Dynamic grouping options
that are pulled from the
log file.
• Ignore List throws out
any line that contains one
of these key words.
• Filter List to only include
lines that contain these
keywords.
• Alert List to
highlight with the red
bulb what you want to
jump out.
20. I Need More Information!
Problem:
Even with all of the new initiatives, proactive support was still difficult:
• We were relying on mining the log files to see what was happening each
day.
• We were unable to link a subscription to the physical report, which was
causing a lot of problems when trying to troubleshoot subscriptions that
were having problems.
• We had no idea how many people were using the system aside from
counting the number of emails in the Saved Mail folder.
We needed even more information!
21. Narrowcast Transmission Recording
• This feature will
create tables in your
Narrowcast
Metadata repository
that log everything
that Narrowcast
does. This includes
details on every
subscription that is
executed.
TN6200-072-0277
22. Enterprise Manager Integration
• For exception reporting and analysis, we need to
extend the Enterprise Manager project to include
the Narrowcast Transmission Data.
• With this you can do
analysis on what has been
happening with
Narrowcast and create
exception reports that
will either email on
occurrence or just give
daily status updates.
23. Narrowcast Schema
This isn’t a comprehensive
schema and only includes
the items that were
necessary for our project.
You could also easily replace
a complicated data model
with free form SQL
reports written to serve
specific purposes.
24. Reports - Execution Log
This report is able to perform a lookup to the iServer metadata and retrieve
the name of the Report that is subscribed to. This is information that is not
available in any Narrowcast Log files or the Administration console. It is
invaluable for tracking down errors.
25. Reports – Subscription Log
This report includes the Title that the user gave when setting up the subscription.
This can help to differentiate between multiple subscriptions with the same name.
This information is mined out of the Subscription Preferences XML stored in the
Subscription Book Repository.
26. Reports – Other Reports
These are some additional reports that were created in this project that are
used daily.
27. Reports – Other Reports
Narrowcast Status Report
Daily quick overview of how many subscriptions ran yesterday.
28. Reports – Other Reports
Narrowcast Daily Failures
A list of subscriptions that failed to deliver today, usually due to excessive
file size or incorrect email addresses.
29. Reports – Other Reports
Deleted Narrowcast Users
Users that were deleted from iServer but still have active subscriptions in
Narrowcast.
30. Reports – Other Reports
Subscriptions to Deleted Objects
Users that have subscriptions to reports that no longer exist.
31. Timeouts 2: The Sequel
Problem:
We started to have timeout problems again, but this time no amount of
teasing the execution settings was helping to resolve it. Our project
had grown and since our schedules required that all subscriptions run at
the same time, this resulted in ~1,000 jobs trying to run at the same
time and bottlenecking.
Solution:
Staggering the services seemed to help resolve the issue by lightening
the load at any one time. But this provided another problem … if we
stagger the schedules, and ETL runs long, then we are sending out
subscriptions before the warehouse is finished loading!
32. Narrowcast Triggers
NCTriggers is a command line program that utilizes MicroStrategy
Command Manager to remotely execute Narrowcast schedules.
Subscriptions are executed based on timing built into the application and
called by a completed ETL system.
The business need for this application is to stagger the execution of
subscription sets so that they aren’t all run at the same time. This
eases the load on Narrowcast server and gives you full control over when
the subscriptions are executed.
33. Narrowcast Triggers - Components
An XML file is passed as a command line property to the program to support
multiple instances from different projects. The file contains the Command
Manager SCP script file which has the actual triggers to execute the Narrowcast
service, the number of minutes it should wait before executing the next item,
and the schedule it should run on.
34. Narrowcast Triggers – Log Files
The left file is an example of the program’s log that is created. This helps in
checking later in the morning what it tried to run. The right file is the
standard Command Manager output text which shows each service that is
executed.
35. Narrowcast Triggers – Notification
When NCTriggers is finished executing a job, it will scan the Command
Manager output file for any errors. If any are found, an Email notification
is sent to the administrative email address with the text.
36. Takeaways
If you only learned three things:
• Knowledge is Power: Proactive alerts and solving issues before they’re real issues. Build
a system that will call for help when it needs it.
• A Happy System is a Stable System: Make sure the proper configurations are applied
for your environment and monitor that everything is executing when it should and in
the time frame it should.
• The Job is Never Done: There is always more to tweak, stay on top of, and analyze when
it comes to a growing environment. Either by analyzing common table hits to find
aggregate opportunities or improving ways that you troubleshoot and diagnose an issue,
there is always room for improvement.
“If it’s worth doing, it’s worth building a tool to do it.”
37. Thanks for Watching
Bryan Brandow
Sr. MicroStrategy Developer
Bluegreen Corp.
bryan.brandow@bluegreencorp.com
January 16th, 2008
www.bluegreencorp.com
Notes de l'éditeur
PLEASE NOTE:The following presentation contains MicroStrategy customizations and configurations based on what worked for our problems in our project. These are shown for your information and entertainment and may not directly translate to solutions for your environment. This presentation is a collection of stories and potential solutions and mostly a heads up that these kinds of things are possible with the right way of thinking. Any action steps or code provided are simply informational bits from our implementation and result from extensive research and trial and error. Before implementing in your solution, please carefully review the proposed modifications, test thoroughly in development, and ensure that you do the proper research via the Knowledge Base and production manuals to know what you’re getting yourself into.Thanks and Enjoy,Bryan
http://www.bluegreencorp.com/
The Marketing Sales Report contains the following features: Each number is a hyperlink that uses the URL API to pass prompt answers along with information on the row and column to a drill report that shows details about the records that make up the number selected. A link at the bottom of the report, labeled “Analyze Report”, takes the user to a grid that matches the data shown on the screen. This allows the user to have all of the report manipulation features available in Grid mode. The Grouping toolbar includes a drop down box if the user ran the report for multiple sites.* Note: All data shown in screenshots are mocked up (random numbers generated in Excel).
Detailed steps:Open Narrowcast Administrator and expand the “Applications” tree.Navigate to the web delivery service and open the service in the Service Wizard.Edit the message body (both text and HTML where applicable) and insert the Subscription Information Objects. subscription info: login idsystem info: service namesubscription info: sub set name
Warning: This modification requires an all or nothing commitment. Since Narrowcast doesn’t currently collect the email address of the owner, you’ll need to supply it via a different field. If you don’t supply it for every user, then a user will be unable to directly reply to an email. You’ll need to make it a common practice to add this configuration whenever creating new users.Detailed steps (courtesy of the MicroStrategy Support Team):1. Edit the Narrowcast user in the Subscription Book Repository which is linked to the MicroStrategy Web user who is signing up services.2. In the 'User Information' tab, populate an unused user information property, such as Zip code or Country or Suffix, with the desired Reply-to email address.3. Edit the two Web Deliveries services for the project, and modify the Control Document.4. Within the control document, you can specify ReplyToEmailAddress='XXX' and ReplyToDisplayName='XXX'. In place of the XXX you can put a dynamic content IO which pulls the user information field you set the email address to, like zip code.After these modifications, whenever a web delivery email is sent out, the service will pull the user information specified in the dynamic content IO and use it as the reply-to information. The From/Sender information will be the information specified for the SMTP ITM. Steps 1 and 2 should be performed for any user you want to utilize web deliveries, otherwise the reply-to information will be 'Error: Information object returned no data.'
Detailed steps:Open Narrowcast Administrator and expand the “Applications” tree.Expand the “Services for Web Deliveries” folder.Locate the “Services for project xxxx” for the project you want to customize.Expand the “Email Delivery” folder.Both Document and Report Email Delivery folders contain a “Service for Report (or Document) Email Deliveries”. This is the service you need to edit with the Service Wizard. Right click the service and choose “Edit with Service Wizard”.Select the “Message Contents” link from the top red bar (Third option).Expand the “Edit Message Body (HTML)” and EditThe Text body does not contain report data from a Web Deliveries, so the HTML body is the only one you need to edit.Locate the proper Information Object that contains the report data. If you’re not sure, replace the error message with something like “First”, “Second”, and so on and run a sample to find out which one is which.Choose the radio button for “Fail the Document”, or alternatively replace the text with the appropriate message.Save your changes, run your service, and see your new Information Objects in action.
For more information regarding Narrowcast Server, consult the Product Manuals, MicroStrategy Knowledge Base, or Google.
Priority:When more jobs are in the queue than available execution slots, Narrowcast will run higher priority jobs (lower number) first. In our environment, we have a multipart executive summary report that goes out first. Since other reports from other projects could be running when this is called, we want to ensure that this takes priority so executives get their numbers as soon as possible. Alternatively, we have general admin reports such as the Daily Narrowcast Status report. It wouldn’t make sense to run this report while other reports are executing, so we set this to a priority 10 (lowest) so that it is the last report to go out. This also signifies that the jobs are all complete.Segment Size:This is the total number of jobs per segment when an execution begins. Narrowcast uses segments as a way of managing large subscription lists and load balancing. If a segment contains 100 jobs and it fails over, all 100 jobs will start to run again. To make things more manageable, we found it advantageous to run with a smaller segment size (10) than the default (100).Segment Timeout:As relative to the previous setting, if you have 100 jobs in a segment (default), each job takes 2mins to run, but your segment timeout is set to 1 hour (default) then your segments will never finish. Even if they are executing perfectly fine, if they’re not done by the timeout setting, they’ll fail over and start again. Make sure that this setting is set appropriately depending on your segment size.Work Allocation:-Determines the number of segments to run simultaneously. Each one will become a job on the iServer, so be sure that this number is lower than your maximum concurrent jobs setting on iServer.
I personally only use the Segment Timeout notification and DM/EE failover notification. Other than those, I do error reporting from my own Enterprise Manager reports shown later in the presentation.Detailed steps:Open Narrowcast Administrator and select the Tools > Set Administrative Notification menu option.Enter a valid email address into the space provided.Check the box for the Message Type you want to activate, and click the “Governing and Filtering” checkbox.Configure the options on the screen. Refer to the Narrowcast Administration Guide for detailed explanations of every option.
Detailed steps:Open Narrowcast Administrator and expand the “Administration” tree.Expand Information TransmittersRight click on Email (SMTP) and select “Edit”Select the “Show Advanced Properties” checkbox at the bottom of the form.Select the “Message Output” tab (third).This screen has the options for setting the Saved Mail location.* NOTE: Be sure that you clean this folder out periodically, as it can get pretty full pretty quickly and cause some server problems. I wrote a simple little VB.NET application that is set to run by the Windows Scheduler once a week to delete emails older than 7 days.
Setup of this feature will vary based on your enterprise email software. Contact your local helpdesk for information on setting up this feature in your company.
Detailed steps:Open Narrowcast AdministratorClick the “Tools” menu and select “View System Messages”Inside the System Message Viewer, select the “Options” menu and choose “Edit Governing Settings”Configure this screen as you see fit. I personally raised the log file size high and set the retention to number of days and not files.
This is a custom application that I wrote in Visual Basic.NET to help with reading and analyzing the huge Narrowcast Log files.
Grab the popcorn.
“Man's quest for knowledge is an expanding series whose limit is infinity” - Stanislaw Lem, His Master's Voice
See TN6200-072-0277 for more information.Detailed steps:To enable Transmission Recording, carry out the following: Via the Narrowcast Administrator console, access the System Governing Settings from the System Menu. Go to the Recording Tab Click on Start and then OK. Note that only services that start after the Transmission recording feature is enabled will record information.
Technically, my Narrowcast solution lives completely segregated inside the Enterprise Manager project. These attributes and facts won’t interact with any of the IServer ones. This is just a clean place to put them. If you don’t own Enterprise Manager (or don’t use it, shame on you), then you can still build your own project with this idea. It really has nothing to do with Enterprise Manager.
The actual data relationships I use are much more complicated than this, and this diagram has been simplified for logical reference. I don’t think this physical model would actually work. If you try this yourself, it involves a lot of iterative processing to make it work for your needs.
SQL Code to extract the Report Title:select sp.mr_sub_guid, t.mr_pref_obj_id, substring(mr_pref_obj, charindex('value=', mr_pref_obj)+7, charindex('/>', mr_pref_obj)-charindex('value=', mr_pref_obj)-8) as titleFrom narrowcast..MSTRPREFOBJS tjoin narrowcast..mstrsubpref sp on t.mr_pref_obj_id = sp.mr_pref_obj_idwhere mr_pref_obj like '%subject name%'
* This is a freeform SQL report.Deleted Narrowcast Usersselect mr_user_name from NarrowcastMetadata..mstrusers where mr_user_id not in(select object_id from iServerMetadata..dssmdobjinfo) and charindex('(', mr_user_upname, 1) > 0 –this is so that it will only check users created via Web Deliveries.
* This is a freeform SQL report.Subscriptions to Deleted Objectsselect r.mr_user_name, mr_address_name, sub_set_name, p.object_name as projectfrom( SELECT mr_channel_id, A2.MR_SUB_SET_ID, A5.MR_OBJECT_NAME AS SUB_SET_NAME, A2.MR_SUB_GUID, A3.MR_USER_NAME, A4.MR_ADDRESS_NAME, A4.MR_PHYSICAL_ADD, A1.MR_PROP_VALUE AS CONTENT_MD_ID FROM narrowcast..MSTREXTENDEDPROPS A1, narrowcast..MSTRSUBSCRIPTIONS A2, narrowcast..MSTRUSERS A3, narrowcast..MSTRADDRESSES A4, narrowcast..MSTROBJNAMES A5 WHERE A1.MR_OBJECT_ID= A2.MR_SUB_GUID AND A3.MR_USER_ID=A2.MR_USER_ID AND A4.MR_ADDRESS_ID=A2.MR_ADDRESS_ID AND A5.MR_OBJECT_ID=A2.MR_SUB_SET_ID AND A1.MR_PROP_ID = 'objectId' AND A2.MR_SUB_SET_ID IN (SELECT MR_DEPN_OBJID AS MR_SUB_SET_ID FROM narrowcast..MSTROBJDEPN WHERE MR_INDEP_OBJID in('99B7D294748A4EACA8C940562186C9EB', '9083737537374D4B8A775E32D5F76593', 'DB75F8E9BD0A4022814293F2900FE3D2', '6812CDEDDFDF475A8762A94C03B2F0EA') AND MR_DEPNOBJ_TYPE=17) –replace with your object id’s for your web delivery services.) rleft join iServerMetadata..DSSMDOBJINFO m on r.content_md_id = m.object_id and r.mr_channel_id = m.project_idleft join iServerMetadata..dssmdobjinfo p on r.mr_channel_id = p.object_idwhere m.object_id is nullorder by 1
I’m sure there is a good “SQL” pun that could go here.
If nothing else, this program makes managing multiple project subscription sets pretty easy. If we want to disable the subscriptions from one project or run them again, we can control it with just a single batch file.
Pay no attention to the XML file behind the curtain.
Upon completion of the job, the application will sleep for an additional 1 minute to allow Command Manager to finish calling the final service. It will then scan the log for today’s date in search of the text “Error”.