BRE Deep Dive

1
Sponsored & Brought to you by
BRE Deep Dive
Johann Cooper
http://www.twitter.com/JohannCooper
https://nz.linkedin.com/in/johanncooper

2
Author:
Integration Monday
BizTalk BRE Deep Dive
Johann Cooper
Johann Cooper – Principal Integration Consultant
@JohannCooper

Copyright © Datacom Group Limited 2014 Saturday, 18 July 2015
About me
• Principal Integration Consultant at Datacom NZ
• Co-author of the book “SOA Patterns with BizTalk
Server 2013 and Microsoft Azure – Second Edition”
published by Packt
• Author of the blog “Adventures inside the Message
Box”
• Creator of the “BRE Pipeline Framework” CodePlex
project
• Author of the White Paper “The A-Y of running
BizTalk Server in Microsoft Azure” published by
BizTalk 360
• Proud dad 

Agenda / Goals
1. Basics of BRE
2. Advanced BRE
3. BRE Pipeline Framework
Goal – To illustrate/evangelize how the usage of the BRE and the BRE Pipeline
Framework can help you design and develop applications which are flexible to
change
Disclaimer – I haven’t used every feature that I’m discussing today in production
applications, and in some cases I’m forced to dumb down the topic in the interest
of time. Feel free to discuss or counter any of my statements post the presentation

Source code
• All source code (BT 2013R2/Visual Studio 2013) and sample files can be
downloaded from the below link
• https://drive.google.com/file/d/0B9cj1Xr--
0vFai0xRHI2aXJnbU0/view?usp=sharing

Why should you use the BRE?
• Decouple your business or processing logic from pipeline/orchestration
implementation
• It’s not just for business rules, you can use it for application logic as well
• It’s fast!
• Easy to change at runtime without full deployments
• Is versioned and can be source controlled
• Rules should be easy to understand if vocabularies are created correctly
• It’s part of the BizTalk Server license, maximize your investment
• Even a business analyst can do it
1 - Basics of BRE

Facts
• Discrete piece of information whose characteristics are to be assessed
and updated
• Supported types
• .NET
• XML
• Database
1 - Basics of BRE

Vocabularies
• Allow you to abstract away implementation of fact characteristics with friendly terms
• Example vocabulary definition – GetColour : “The primary colour of the cow”
• .NET – cow.Color
• SQL – select [Color] from dbo.Cow
• XML - /cow/color
CowVocabulary (Name – Display Name)
• GetColour – The primary colour of the cow
• GetLength – The length of the cow in meters
• SetCowValue – Update the cow’s value to ${0}
1 - Basics of BRE

Vocabularies
DEMO
1 - Basics of BRE

Rules
• Made up of conditions and actions
• String together fact/vocabulary definitions with relevant predicates
• Example rule – Set value for large green cows
• Condition
• If the cow’s length in meters is greater than 500
• And the cow’s primary colour is equal to Green
• Action
• Update the cow’s value to $1000000000
1 - Basics of BRE

Rules
• You don’t have to use vocabularies but why wouldn’t you?
• The below is just as valid.
• Example rule – Set value for large green cows
• Condition
• cow.Length is greater than 500
• cow.color is equal to Green
• Action
• SetCowValue 1000000000
1 - Basics of BRE

Policies
• A collection of rules
• All conditions will be evaluated first, filling out the agenda, before any rules fire
• Order of conditions being evaluated is non-deterministic
• Order of rule actions firing is non-deterministic unless priorities are specified
• Order of actions firing within a rule is based on order inside the rule
1 - Basics of BRE

Policy stages
1. Match – condition evaluation and adding rules to agenda
2. Conflict resolution – prioritizing sequence of execution
3. Action – Firing actions in the relevant order
1 - Basics of BRE

Assertion
• Making a fact available to a policy
• A policy can only deal with facts it is aware of
• Rules that depend on facts that are not asserted will never fire
• It is possible to enable static support for static .NET method via registry key
• HKEY_LOCAL_MACHINESOFTWAREMicrosoftBusinessRules3.0StaticS
upport
• 0 is default value which implies no static support
• 1 implies static support, with the static method being called each time
• 2 implies static support with the result of the static method being cached
(only for conditions, not actions) and reused after the first execution.
1 - Basics of BRE

RETE Algorithm
• Very fast, and efficient way of
evaluating multiple rules and
creating your agenda
• Sacrifices memory for speed
• Breaks apart left and right hand
side of conditions into separate
node networks and applies the
minimal amount of joins to
evaluate conditions
• Might be best left to PhDs 
1 - Basics of BRE

Rule Priorities
• Priorities can be assigned to rules
• Default value 0
• Priority has nothing to do with conditions, just actions
• Higher the value, higher the priority
• Higher priority rules fire before lower priority rules
• So lower priority rules can override higher priority rules!
1 - Basics of BRE

Policy Caching
• CacheEntries, CacheTimeOut, and PollingInterval can be configured via Registry or
BizTalk config file - https://msdn.microsoft.com/en-us/library/aa548008.aspx
1 - Basics of BRE

Side Effects
• Once you access an XML or SQL based fact value in a policy, it’s value will be
cached and reused in other conditions/actions
• .NET members will not be cached by default
• This is controlled by the sideeffects property, can only be set programmatically or
via XML export files, no support in composer
1 - Basics of BRE

Side Effects
• This is a deep topic - further reading
• Charles Young -
http://geekswithblogs.net/cyoung/archive/2007/04/09/111169.aspx
• My blog -
https://adventuresinsidethemessagebox.wordpress.com/2012/11/20/biztal
k-business-rules-policies-not-caching-net-facts-by-default/
1 - Basics of BRE

Rules/Policies/Assertion/Priorities
DEMO
1 - Basics of BRE

Rules testing
• Can right click on policy version in composer and choose “Test Policy”
• Can assert XML files or database connections via wizard
• Testing policy will result in change in state in asserted facts
• Must use a fact creator to assert .NET objects
• There is also a BizUnit 4.0 FactBasedRuleEngineStep test step
2 – Advanced BRE

Rules testing
DEMO
2 – Advanced BRE

Rules traceability
• Can get equivalent information at runtime as you would when choosing
“Test Policy”
• Can only do this programmatically, not in orchestration
• Can build your own tracing class based on Microsoft.RuleEngine.
IRuleSetTrackingInterceptor
• Alternatively can make use of out of the box
Microsoft.RuleEngine.DebugTrackingInterceptor
2 – Advanced BRE

Rules traceability
DEMO
2 – Advanced BRE

Versioning and Source Control
• Can export vocabulary/policy definitions to XML using BRE Deployment Wizard
or BizTalk Administration Console
• Can bulk export from BizTalk Administration Console
• Can copy old versions of vocabularies or policies and paste to create a new
version and edit
• Orchestration always chooses latest version, but can be explicit from .NET (or
ESB Toolkit / BRE Pipeline Framework)
• Can associate policies with an application, but beware that deleting the
application is equivalent to deleting the policy
• If deleting a policy from BizTalk Admin Console, if it is the last policy to be
associated with a vocabulary then the vocabulary will be deleted as well
2 – Advanced BRE

Versioning and Source Control
DEMO
2 – Advanced BRE

Engine Control Functions
• Can control rules chaining through definitions contained within the Functions
vocabulary
• Beware of endless loops, these can consume all the machine’s memory
• This is a topic in itself. We are only scratching the surface
• Assert/Update – Re-evaluate conditions in all rules that make use of the
asserted/updated fact. Update will only re-evaluate if the type is used in
conditions, not if only in actions (in which case the actions will be left on the
agenda)
• Retract/RetractByType – Remove fact from rule engine. RetractByType removes
all facts with a matching type while retract only removes a single instance
• Halt – Stop the rules execution (tip – use priority to control when this happens)
2 – Advanced BRE

Engine Control Functions
DEMO
2 – Advanced BRE

Long Running Facts
• Can be implemented using an IFactRetriever interface implementation
• UpdateFacts method can be used to assert/retract one or many facts which will
be cached for use until they are retracted
• Can return a token object out of UpdateFacts that will be received into the next
firing of the method which will can be used to decide whether to retract/re-
assert the long running fact
• FactRetriever is associated with a policy version so no need to assert it from
orchestration/.NET code
• Pro Tip – Can be used to assert a .NET helper class and use contained methods
within the policy
2 – Advanced BRE

Long Running Facts
DEMO
2 – Advanced BRE

Deployment
• Multiple deployment options
• BizTalk MSI
• BRE Deployment Wizard
• BTDF
• BizTalk Administration Console
• With BizTalk MSIs and BizTalk Administration Console you will need to install
.NET assemblies into GAC first, restart the Administration Console and then
perform the import.
• .NET based vocabularies with long namespace/class names can cause BizTalk
MSIs to fail
• Might want to consider deploying policies, restarting Rule Engine Update Service
and then starting application
2 – Advanced BRE

BTDF BRE Sections
<PropertyGroup>
…
<IncludeVocabAndRules>True</IncludeVocabAndRules>
…
<PropertyGroup>
<ItemGroup>
…
<RuleVocabularies Include="Vocabulary.1.0.xml">
<LocationPath>..BRE ArtifactsVocabularies</LocationPath>
</RuleVocabularies>
<RulePolicies Include="Policy.1.0.xml">
<LocationPath>..BRE ArtifactsPolicies</LocationPath>
</RulePolicies>
…
<ItemGroup>
2 – Advanced BRE

Updating published/deployed vocabularies and
policies
• Change nStatus to 0 on relevant tables in BizTalkRuleEngineDb database
• You can just publish vocabularies as you would normally
• Won’t be able to publish/deploy a rule that you’ve unpublished from the
composer. You’ll need to save it and then set nStatus back to 1
• Be cautious when doing this, and please restrict this to dev!
2 – Advanced BRE

Working out dependencies
2 – Advanced BRE

BRE Pipeline Framework Introduction
• A pipeline component which allows you to execute BRE Policies
• Provides a range of utility vocabularies that help you implement lots of
common integration patterns
• Is extensible so you can implement your own utility and corresponding
vocabularies
• Makes it easy to assert XML and SQL database facts
• Encourages messaging only application where orchestration would be used
for utility rather than orchestrating a process
3 – BRE Pipeline Framework

Some assurances on reliability
• It’s compatible with BizTalk 2010+, same code base/installer
• It’s been heavily unit tested on all compatible versions of BizTalk – 94% code
coverage
• Theoretically no reason it won’t continue to work with new versions of
BizTalk
• New versions in the 1.x version set will always be backwards compatible
• Close to 1000 downloads (and only about 30 of those are me ), is heavily
used in NZ (not just in the consultancy I work for)
• Been successfully used on low latency and high throughput solutions as well
as solutions dealing with large messages

Installation + Demo
• Download installer from CodePlex site -
http://brepipelineframework.codeplex.com/
• Uninstall previous version (if applicable)
• Install new version
• Restart host instances (if previous version)
• Import OOTB vocabularies that you’re interested in from “C:Program Files
(x86)BRE Pipeline FrameworkVocabularies”
• Add pipeline component to pipeline toolbox in Visual Studio on dev
machines (right click in toolbox and click on “Choose items”)

Some important terms
• ExecutionPolicy – BRE Policy used to assess conditions and update a
message’s body or context
• InstructionLoaderPolicy – Optional BRE Policy used to instantiate and assert
facts to the ExecutionPolicy
• Used for custom .NET facts, SQL facts or XML facts
• Not needed to use OOTB facts
• ApplicationContext – Used to instruct the BRE Policy which pipeline (and
optionally pipeline stage) called the Policy

Pipeline component and simple rules
DEMO

Some more important terms
• MetaInstruction
• Class that contains helper methods that you can use in conditions or
actions to get details from a message
• Almost all BRE Vocabularies provided in the framework correspond to
MetaInstruction, and can only be used in ExecutionPolicy
• Also contains methods you can use in actions that add an instruction
• Instruction
• Class that is used to update a message

Pipeline Component Architecture

BREPipelineFramework.SampleInstructions.ContextIns
tructions Vocabulary + Demo
• Get or set context properties
• By enumeration for all out of the box context properties
• By explicitly specifying property namespace and name
• Set context property from SSO config store key/value pair or XPath query
against message body
• Remove context property

BREPipelineInstructions.SampleInstructions.HelperInst
ructionsVocabulary
• String manipulation – ToLower/ToUpper, Regex/Substring Find/Replace,
StringContains, Concatenate
• Dynamic transformation
• Pro Tip – you can use the Context Accessor on a send port as well when
you use this function
• Run XPath queries to extract node name, namespace or value
• Nullify message
• Generate GUID

BREPipelineInstructions.SampleInstructions.HelperInst
ructionsVocabulary + Demo
• Round up time (great for convoys or rolling log file naming)
• Tracing
• Throw exceptions
• Get Value from SSO Config Store
• Casting

BREPipelineFramework.SampleInstructions.CachingIn
structions
• Uses .NET MemoryCache class which is specific to an AppDomain
• Recommendation is to only rely on cached context between a receive and send
pipeline on a given two way port. Don’t rely on it across ports as a host
instance restart would result in the cache being emptied, and this also wouldn’t
work on a BizTalk group with multiple runtime servers
• Cached items automatically expire after 30 minutes, or you can remove them
explicitly. By default items will be removed from the cache if the server is
under heavy memory pressure but you can override this too

BREPipelineFramework.SampleInstructions.CachingIn
structions + Demo
• Cache all or specific context properties to .NET MemoryCache
• Reapply all or specific context properties from cache that is relevant to the
current message
• Get cached context property that is relevant to the current message
• Cache or get any custom string for which you must provide a key
• Get a value from an SSO config store and also cache it, subsequent reads will be
from the cache

BREPipelineFramework.SampleInstructions.HttpHeade
rInstructions + Demo
• Get values of individual inbound HTTP headers
• Set values for individual outbound HTTP headers
• Copy individual inbound HTTP headers to outbound HTTP headers
• Pro tip – Set the BTS.IsDynamicSend context property to true to dynamically
set outbound HTTP headers on a static WCF-WebHttp send port
• Along the same veins, support for content negotiation in BizTalk 2013 R2 -
https://adventuresinsidethemessagebox.wordpress.com/2015/06/30/dyna
mic-request-content-and-response-content-negotiation-for-restful-apis-
exposed-by-biztalk-server/

BREPipelineFramework.SampleInstructions.PipelineIns
tructions + Demo
• Execute BizTalk Pipelines
• XML Assembler/Disassembler
• XML Validator
• Flat File Assembler/Disassembler
• Note that disassembly of an envelope containing multiple bodies will result
in only the first body being passed to the next pipeline component. The
BRE Pipeline Framework is not a disassembler (yet)!

BREPipelineFramework.SampleInstructions.XMLTransl
atorInstructions
• Modify XML message in a streaming manner
• Add/Replace/Remove namespace/prefix on specified nodes
• Update element/attribute names/values on specified nodes
• Remove element (and optionally all child elements)

BREPipelineFramework.SampleInstructions.PartInstruc
tions
• Get/Set message part properties by part name or index
• Get/Set message part content type/char set by part name or index
• Get part names by index
• Get part count

Asserting XML Facts
• Use SetTypedXmlDocument definition in BREPipelineFramework vocabulary
to set the document type (must match document type in XML vocabulary
definition)
• Can conditionally set the document type based on ApplicationContext, root
node name and namespace, regex queries, string finds etc… These can all
be found in the BREPipelineFramework vocabulary and besides
ApplicationContext, they can only be used in an InstructionLoaderPolicy
• Once asserted, XML vocabularies can be used in the ExecutionPolicy as per
normal
• By default all XML fact based actions will occur before any other
instructions, but this behavior can be overridden
• Can only assert one XML fact (since there’s only one XML message body)

BREPipelineFramework.SampleInstructions.XMLInstru
ctions + Demo
• Manipulate XML facts
• Add element/attribute in a specified location
• With or without namespace
• With or without value

Asserting SQL Facts + Demo
• Currently only DataConnection based facts are supported, not
TypedDataTable or TypedDataRow
• Use AddSQLDataConnection or AddSQLDataConnectionFromSSO definitions
in the BREPipelineFramework vocabulary to assert SQL facts in an
InstructionLoaderPolicy
• Can assert as many SQL Facts as you want
• Use the SQL Facts as you would normally within your ExecutionPolicy
• Can assert the facts conditionally, as previously described for XML facts

Extensibility - Instruction
• Your instruction class must implement the
BREPipelineFramework.IBREPipelineInstruction from the
BREPipelineFramework assembly (can be found in “C:Program Files
(x86)BRE Pipeline FrameworkBin” or GAC)
• Only need to implement one method called execute, similar signature to a
pipeline component’s execute method
• void Execute(ref IBaseMessage inmsg, IPipelineContext pc);
• Feel free to add whatever properties and constructors you need to the class
• Modify the message to your heart’s content though keep normal pipeline
component considerations in mind (streaming, adding to pipeline context
resources etc…)

Extensibility - MetaInstructions
• Derive your MetaInstruction class from
BREPipelineFramework.BREPipelineMetaInstructionBase base class found in
BREPipelineFramework assembly
• The class will always be instantiated with a default constructor
• Build any helper methods you want with any signatures, and add any properties to the
class as well
• You can access the message via base.inMsg and pipeline context via base.pc
• You will not be able to, and should not attempt to modify a message in a
MetaInstruction
• You can build methods that instantiate Instruction instances, and then call
base.AddInstruction method to add the instruction
• Build vocabularies for methods you want to use in your policy

Asserting your MetaInstruction + Demo
• Two methods of asserting your MetaInstruction
• Add fully qualified MetaInstruction class/assembly name to the
CustomMetaInstructionSeparatedList parameter on pipeline component.
This is a semi-colon separated list, so you can assert as many
MetaInstructions as you want
• You can use the AddMetaInstruction definition in the
BREPipelineFramework vocabulary in an InstructionLoaderPolicy to
conditionally assert MetaInstructions
• Only use this method if it is really important that you conditionally
assert your MetaInstruction. Avoiding having to maintain an
InstructionLoaderPolicy is a huge boon
• You can then use vocabularies corresponding to methods in your
MetaInstruction just as you would with the OOTB vocabularies

Rules Traceability + Demo
• You can use the TrackingFolder parameter on the pipeline component which
will write out rules debug information to a nominated folder, similar to
choosing “Test Policy” in the composer
• You can use the CAT Instrumentation Framework Controller to enable Pipeline
Component traces. ETW tracing features heavily throughout the BRE Pipeline
Framework and will give you deep insight into what is going on as well as
execution time

Things I haven’t touched upon today
• You can define constant based vocabulary definitions as well, including lists
• ESB Toolkit BRE Resolvers haven’t been discussed since there has been a
recent session on the ESB Toolkit
• Can assert SQL Facts based on TypedDataTable or TypedDataRow instead of
DataConnection. If you want to build a long running SQL based fact then
you must use these
• Can execute policies via RuleEngine.Execute() instead of Policy.Execute()
which allows you to resume halted rules execution
• Complex/repeating XML structures -
https://adventuresinsidethemessagebox.wordpress.com/2012/05/23/biztal
k-business-rules-and-complex-xml-structures/

Q&A
• Any questions?

BRE Deep Dive

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

En vedette

En vedette (12)

Similaire à BRE Deep Dive

Similaire à BRE Deep Dive (20)

Plus de BizTalk360

Plus de BizTalk360 (20)

Dernier

Dernier (20)

BRE Deep Dive

Notes de l'éditeur