Successfully reported this slideshow.
MT-Logging with WSO2 BAM                         Amani Soysa             Senior Software Engineer
How it Works??Logs are published to BAM via Log4JAppenderPublished Logs are Stored in Cassandra (per tenant per day)When t...
Log Publisher - Log4JAppender
Setting Up LogEventAppender    log4j.rootLogger=INFO, LOGEVENT# LOGEVENT is set to be a LogEventAppender using a PatternLa...
Stream DefinitionstreamId = dataPublisher.defineStream("{" + "name:log" + "." + tenantId +"."+ severName + "." + currDateS...
Log Analyzer
Logging-Config.xml    Basic Information        <isDataFromCassandra>true</isDataFromCassandra>    Information Regarding ...
Logging SummarizerArchive logs daily at a given time and store in hdfs file system
summarizer-config.xml    Basic Information        <isDataFromCassandra>true</isDataFromCassandra>        <deleteColFamily...
Advantages      Asynchronous and None Blocking Data    publishing      Receives and Stores Log Events Cassandra    Clust...
Log Viewer    Get Logs From Cassandra - <<Latest>>    Get Archived Logs From HDFS File system
Log Viewer – Archived Logs
Log Viewer – Application Logs
Mt logging with_bam
Mt logging with_bam
Prochain SlideShare
Chargement dans…5
×

Mt logging with_bam

483 vues

Publié le

WSO2 BAM 2.0 provide a rich set of tool for aggregation, analyzing and presentation for large scale data sets and any monitoring scenario can be easily modeled according to the BAM architecture. We selected WSO2 BAM as the backbone of our logging architecture with a Log4JAppender to send LogEvents to bam.

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Mt logging with_bam

  1. 1. MT-Logging with WSO2 BAM Amani Soysa Senior Software Engineer
  2. 2. How it Works??Logs are published to BAM via Log4JAppenderPublished Logs are Stored in Cassandra (per tenant per day)When the day is over Logs are published to HDFS file systemdaily using BAM analyzersTo view Logs :-Application Logs – Log events on web applications/services filterby the application nameSystem Logs – View the system log of each product {Withfiltering categories}Log Viewers get logs from cassandra <<real time log>> andHDFS file system <<Archive logs>>
  3. 3. Log Publisher - Log4JAppender
  4. 4. Setting Up LogEventAppender log4j.rootLogger=INFO, LOGEVENT# LOGEVENT is set to be a LogEventAppender using a PatternLayout to send logs to LOGEVENTlog4j.appender.LOGEVENT=org.wso2.carbon.logging.appender.LogEventAppenderlog4j.appender.LOGEVENT.url=tcp://localhost:7611log4j.appender.LOGEVENT.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayoutlog4j.appender.LOGEVENT.columnList=%T,%S,%A,%d,%c,%p,%m,%H,%I,%Stacktracelog4j.appender.LOGEVENT.userName=adminlog4j.appender.LOGEVENT.password=adminlog4j.appender.LOGEVENT.processingLimit=1000log4j.appender.LOGEVENT.maxTolerableConsecutiveFailure=20
  5. 5. Stream DefinitionstreamId = dataPublisher.defineStream("{" + "name:log" + "." + tenantId +"."+ severName + "." + currDateStr + "," + " version:1.0.0,"+ " nickName: Logs," + " description: Logging Event,"+ " metaData:[" + " {name:clientType,type:STRING}" + " ],"+ " payloadData:[" + " {name:tenantID,type:STRING},"+ " {name:serverName,type:STRING},"+ " {name:appName,type:STRING},"+ " {name:logTime,type:LONG},"+ " {name:priority,type:STRING},"+ " {name:message,type:STRING},"+ " {name:logger,type:STRING},"+ " {name:ip,type:STRING},"+ " {name:instance,type:STRING},"+ " {name:stacktrace,type:STRING}" + " ]" + "}");
  6. 6. Log Analyzer
  7. 7. Logging-Config.xml Basic Information <isDataFromCassandra>true</isDataFromCassandra> Information Regarding Cassandra <cassandraHost>localhost:9160</cassandraHost> <userName>admin</userName> <password>admin</password> Information Regarding HDFS Server <archivedHost>hdfs://localhost:9000</archivedHost> <archivedHDFSPath>/stratos/logs</archivedHDFSPath>
  8. 8. Logging SummarizerArchive logs daily at a given time and store in hdfs file system
  9. 9. summarizer-config.xml Basic Information <isDataFromCassandra>true</isDataFromCassandra> <deleteColFamily>false</deleteColFamily> Information Regarding Cassandra<cassandraHost>localhost:9160</cassandraHost> <userName>admin</userName> <password>admin</password> Information Regarding BAM <publisherURL>localhost:7611</publisherURL> <userName>admin</userName> <password>admin</password> <cronExpression>0 0 1 ? * * *</cronExpression> Information Regarding HDFS Server <archivedHost>hdfs://localhost:9000</archivedHost> <archivedLogLocation>/stratos/logs</archivedLogLocation>
  10. 10. Advantages Asynchronous and None Blocking Data publishing Receives and Stores Log Events Cassandra Cluster which is high scalable and a big Data Repository Rich tools set for analytics Can be shared with CEP for real time Log Event analysis. Can provide Logging tool boxes and dashboards for system administrators using WSO2 BAM
  11. 11. Log Viewer Get Logs From Cassandra - <<Latest>> Get Archived Logs From HDFS File system
  12. 12. Log Viewer – Archived Logs
  13. 13. Log Viewer – Application Logs

×