This session will review significant enhancements to Pipeline Pilot in recent releases. Areas covered are: Professional client, administration, security, integration, databases, and collections (chemistry, next gen sequencing, documents and text, statistics, and imaging).
Salesforce Miami User Group Event - 1st Quarter 2024
(ATS3-PLAT01) Recent developments in Pipeline Pilot
1. (ATS3-PLAT01) Recent developments
in Pipeline Pilot
Ton van Daelen
Sr. Director, Platform Product Management
ton.vandaelen@accelrys.com
2. The information on the roadmap and future software development efforts are
intended to outline general product direction and should not be relied on in making
a purchasing decision.
3. Introduction
• Target audience
– Administrators
– Developers
• What will you learn?
– What is new that affects me or my users (and me indirectly)
• More of What’s New in
– (ATS3-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
• Slides and examples will be made available
5. Pipeline Pilot Release History
• Pipeline Pilot 8.0 – Dec 2010
– Design mode 1-4
– Local subprotocols
– Data flow monitoring 5.0
6.0
• Pipeline Pilot 8.5 – Aug 2011
– Support growing deployments 7.0
– Isentris and Notebook integration 8.0
8.5
– Support for building ‘apps’
• Pipeline Pilot 8.5 CU1 – Nov 2011
– Protocol comparison
9.0
– Performance
6. Science and Applications Release History
Imaging (HCS)
Reporting (Canvas elements)
Advanced data modeling
Chemistry (MDL formats)
Reporting (Stylesheets)
7.5 Imaging (DICOM)
Direct cartridge
Database Search
Imaging (Big Image Explorer)
Chemistry (Content, NEMA, Cheshire)
8.0 Next generation sequencing
8.0
Isentris and Notebook integration
ChemReg
Collections BioReg
Apps
8.5
9. Performance Testing
• Performance increasingly critical because of ‘apps’
– Both Accelrys and custom
• Performance testing
– Track our progress
– Spot errors
• How do we do this?
15. Server Maintenance and Monitoring
• Administration portal
– Run diagnostics
– View and cancel running
jobs
– View log files
– Monitor CPU and memory
• Pro client
– Server status indicator
16. Server Maintenance and Monitoring
• External tools
Windows Performance Counters Ganglia (Linux)
17. Usage Monitoring
• Why?
– Internal accounting (‘Which dept is using imaging?’)
– Justify investment (‘Are we using reporting?’)
– Administrative (‘What protocols haven’t been used in 1 yr?’)
• Approaches
– Usage report in admin portal
– Custom analysis of log files
– Protocol database analysis using Component Reader
• More in: (ATS3-PLAT11) Advanced Planning
19. Usage Report Example
Introduced in 8.5. For pre 8.5 servers, install protocols from
ftp://download.accelrys.com/protocols/SummaryReport.zip.
20. Logfile Folder
Installation logfiles.
Remote logfiles. Allows for
manual copying if server is
not connected Local logfiles.
21. Usage Log Analysis Through Protocols
• Usage protocol in C:Program
FilesAccelrysPPSappsscitegicadmindatarootxml
C:Program FilesAccelrysPPSlogsusagePipeline Pilot.log
22. Analysis Example
Runs by User Runs by ClientType Most Frequent Protocols
23. Analyzing the XMLDB
• Questions to ask
– Non-empty Password parameters?
– Empty required parameter field?
– File references?
– Shortcuts as copies?
– Protocol for a particular user?
– Recent protocols?
– Components of a certain collection?
– Deprecated components?
• Combined with log file
– What components were used recently?
– For a migration, what protocols were run recently?
24. Analyzing the Xmldb
Protocol info:
• Author
• Date
• Component ID
Component info:
• Data source references
• Passwords
28. Protocol Validation
• Inspect protocols before moving to production
• Rules checking:
– Hard-coded passwords
– Bad file of database references
– Broken shortcuts
• Configurable for different deployment modes
– Standard protocol, Reports, Web services, RSS
Feeds, SharePoint, etc.
29. Protocol Validation
• Validation can be set up as hard requirement for publication
• Operation is cancelled on validation errors
*Not a standard feature. Request from support
(ATS3-PLAT03) Enterprise Protocol & Package Deployment
30. Automating Component Best Practices
• Component Review-o-Matic
Protocol
– Checks Documentation
– Checks Parameter Names
• Create your own custom rules
See: ComponentsDatabase and Application
IntegrationUtilitiesPrototypesUsabilityComponent Review-o-Matic
32. Windows Integrated Authentication
• Run Web Port without providing
username /password
• Supported via Kerberos and NTLM
• Requirements
– DOMAIN authentication
– Kerberos only works on remote
servers. Negotiation on local servers
uses NTLM instead of Kerberos, so
some restrictions may apply
– Pipeline Pilot Apache service must
have valid Service Provider Name on
the DOMAIN server. Running as "Local
System" usually fulfills this
requirement
– Configuration required on server and
client’s browser
33. Windows Integrated Authentication
• Requires Active Directory settings
• Enables Secure Services
• Authentication provided by NTLM/Kerberos
• Limitations
– Delegation not supported
– Accessing network resources (e.g., proxy server authentication with
credentials, UNC paths)
– Using PP credentials for components that require authentication
(e.g., database, readers, integration, etc.)
– Running protocols from within a protocol (e.g., RunProtocol
PilotScript function, parallel subprotocols)
34. Server and Client Configuration
• Run Apache service as a user:
– To set up Kerberos service principal name for the Pipeline Pilot server, run the following
on the Domain controller:
• setspn.exe -a http/SERVERNAME DOMAINapache_username
– On the Pipeline Pilot server, use the Services tool and set the Logon User for the
"scitegic_apache_8_5_0" service.
• Client Configuration
– Internet Explorer:
• Add the server as a trusted site (Tools > Internet Options > Security > Trusted Sites > Custom Level >
User Authentication > Logon).
• Select Automatic logon with current user name and password.
• If your server is already part of the Local Intranet, select Automatic logon only in Intranet zone.
– Firefox:
• Browse to "about:config" and add the server names to the following preferences:
– network.negotiate-auth.trusted-uris
– network.automatic-ntlm-auth.trusted-uris
35. Pass-through (Impersonation)
Pipeline Pilot Web Services
Webform Basic
Scientist
User Token
Database
SciServer
ODBC
Web Services
Basic
37. Pipeline Pilot Integration Points
Web Browser
Run Protocol .NET Client Java Client SOAP Client
Professional Client Command Line Web Port JavaScript Client
Client
.NET Client
JavaScript Client SDK Java Client SDK Web Services API
SDK
Pipeline Pilot Enterprise Server
Web Apps Web Services API Admin Portal Help Portal
Protocol Runtime Environment (scisvr)
VB Script VB Script
Run SOAP & Telnet / ODBC /
(On (On Java Perl Python .NET SSH / SCP
Program HTTP FTP JDBC
Client) Server)
Java Perl .NET
VB Script Cmd Line
Classes Scripts Classes
REST /
SOAP
HTTP Cmd Line DBs
Service
Service
38. Component Development Update
• .NET On Server
– Points to a compiled .NET Assembly
• Dynamic .NET
– Embed your .NET code directly into
the Server Component
• Leverage .NET supported
languages
– C#
– VB.NET
– IronPython
43. Pipeline Pilot & SharePoint
Web Services Run Job Web Parts (3)
Pipeline Pilot SharePoint Bridge
User Created
IT Created
Read/Write Document Library
ISV Created
Protocols Lists
Pipeline Pilot SharePoint
45. Deploying Custom Web Parts
• Problem/Need: Want to create Pipeline Pilot-based Web Parts and deploy to
organization
• Solution: Create packaged (“standalone”) Web Parts (Chapter 4, SharePoint
Bridge Admin Guide)
46. Using Custom Web Parts
• Custom Web Parts are
available in Web Part Gallery
• Provides a much simpler
mechanism for end-users
47. SharePoint Document Library
• Read from / write to
document library
– Use standard Reader/Writer
components
– With Domain
Authentication, R/W will
simply work
– Alternatives
• Copy between document
repository and temporary
local version
• Add authentication options to
Reader/Writer/Copy
components
48. Summary
• Reasons to upgrade
– Administrative
– Development
– Science and applications
• Plan your upgrade
– Engage Accelrys support and services
– (ATS3-PLAT11) Advanced Planning
• Resources
– Component development guide
– Accelrys Community: http://community.accelrys.com/
• Other sessions of interest
– (ATS03-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
49. The information on the roadmap and future software development efforts are
intended to outline general product direction and should not be relied on in making
a purchasing decision.
For more information on the Accelrys Tech Summits and other IT & Developer
information, please visit:
https://community.accelrys.com/groups/it-dev