Java for ultra low latency applications
•Télécharger en tant que PPTX, PDF•
1 j'aime•631 vues
Talking about java for ultra low latency apps on The Developers Conference 2017 http://www.thedevelopersconference.com.br/tdc/2017/saopaulo/trilha-java
Signaler
Partager
Signaler
Partager
Recommandé
Troubleshooting RabbitMQ and services that use it
This document discusses troubleshooting RabbitMQ and services that use it. It provides tips for troubleshooting publishers and consumers, including common issues like I/O exceptions, inadequate delivery quality of service, and lack of confirmations. It also recommends collecting logs, metrics, exceptions and traces system-wide to analyze issues. Distributed system debugging remains a challenging problem.
XFLTReat: a new dimension in tunnelling
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Messaging, interoperability and log aggregation - a new framework
In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages.
Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ.
The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports.
They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems.
You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily.
Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed.
Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement.
Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks.
Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.
OFFENSIVE: Exploiting DNS servers changes BlackHat Asia 2014
This document discusses exploiting changes to DNS server configurations to intercept network traffic. It begins by introducing the speaker and their background in security research. Then it outlines normal exploitation procedures like CSRF, default passwords, and rogue DSLAMs. The document explores using tools like Metasploit, Dnsmasq and Bind to hijack DNS servers and redirect traffic. It discusses obstacles like SSL certificates and ways to improve attacks using features like DNS load balancing. The document demonstrates a proof of concept tool called dns2proxy that can spoof and redirect DNS responses. It also discusses defeating protections like HSTS and exploring attacks using protocols like UDP. In conclusion, the document advocates that improved DNS hijacking using multiple tools can capture more information than
Rabbits, indians and... Symfony meets queueing brokers
This document discusses using message brokers like RabbitMQ to introduce queues for processing document conversions. It describes how using a queue allows processing to be distributed across multiple consumers for better reliability and scalability compared to a single process approach. Specifically, it outlines how the Kaliop QueueingBundle for Symfony provides an easy way to integrate queues and supports multiple broker protocols with a common API.
Dock ir incident response in a containerized, immutable, continually deploy...
This document discusses incident response strategies in a containerized and immutable infrastructure environment like Docker. It addresses challenges like lack of system and software inventory visibility due to rapid container changes, and lack of agent-based security due to single-purpose containers. It proposes solutions like establishing managed base container OSs, whitelisting allowed containers and files, and leveraging logs and sidecar containers to monitor for detections. Response challenges around long investigation timeframes due to short container lifetimes and lack of access are addressed with strategies like comprehensive logging, filesystem artifact preservation, and automating remote response capabilities.
Project Basecamp: News From Camp 4
Reid Wightman's presentation at AppSec DC 2012. Reid provides background and the lates on Digital Bond's Project Basecamp. New PLC exploit modules include a Stuxnet-type attack on the Modicon Quantum.
Infrastructure as code might be literally impossible part 2
The document discusses various issues with infrastructure as code including complexities that arise from software licenses, bugs, and inconsistencies across tools and platforms. Specific examples covered include problems with SSL and APT package management on Debian/Ubuntu, Linux networking configuration difficulties, and inconsistencies in Python packaging related to naming conventions for packages containing hyphens, underscores, or periods. Potential causes discussed include legacy code, lack of time for thorough testing and bug fixing, and economic pressures against developing fully working software systems.
Recommandé
Troubleshooting RabbitMQ and services that use it
This document discusses troubleshooting RabbitMQ and services that use it. It provides tips for troubleshooting publishers and consumers, including common issues like I/O exceptions, inadequate delivery quality of service, and lack of confirmations. It also recommends collecting logs, metrics, exceptions and traces system-wide to analyze issues. Distributed system debugging remains a challenging problem.
XFLTReat: a new dimension in tunnelling
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Messaging, interoperability and log aggregation - a new framework
In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages.
Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ.
The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports.
They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems.
You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily.
Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed.
Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement.
Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks.
Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.
OFFENSIVE: Exploiting DNS servers changes BlackHat Asia 2014
This document discusses exploiting changes to DNS server configurations to intercept network traffic. It begins by introducing the speaker and their background in security research. Then it outlines normal exploitation procedures like CSRF, default passwords, and rogue DSLAMs. The document explores using tools like Metasploit, Dnsmasq and Bind to hijack DNS servers and redirect traffic. It discusses obstacles like SSL certificates and ways to improve attacks using features like DNS load balancing. The document demonstrates a proof of concept tool called dns2proxy that can spoof and redirect DNS responses. It also discusses defeating protections like HSTS and exploring attacks using protocols like UDP. In conclusion, the document advocates that improved DNS hijacking using multiple tools can capture more information than
Rabbits, indians and... Symfony meets queueing brokers
This document discusses using message brokers like RabbitMQ to introduce queues for processing document conversions. It describes how using a queue allows processing to be distributed across multiple consumers for better reliability and scalability compared to a single process approach. Specifically, it outlines how the Kaliop QueueingBundle for Symfony provides an easy way to integrate queues and supports multiple broker protocols with a common API.
Dock ir incident response in a containerized, immutable, continually deploy...
This document discusses incident response strategies in a containerized and immutable infrastructure environment like Docker. It addresses challenges like lack of system and software inventory visibility due to rapid container changes, and lack of agent-based security due to single-purpose containers. It proposes solutions like establishing managed base container OSs, whitelisting allowed containers and files, and leveraging logs and sidecar containers to monitor for detections. Response challenges around long investigation timeframes due to short container lifetimes and lack of access are addressed with strategies like comprehensive logging, filesystem artifact preservation, and automating remote response capabilities.
Project Basecamp: News From Camp 4
Reid Wightman's presentation at AppSec DC 2012. Reid provides background and the lates on Digital Bond's Project Basecamp. New PLC exploit modules include a Stuxnet-type attack on the Modicon Quantum.
Infrastructure as code might be literally impossible part 2
The document discusses various issues with infrastructure as code including complexities that arise from software licenses, bugs, and inconsistencies across tools and platforms. Specific examples covered include problems with SSL and APT package management on Debian/Ubuntu, Linux networking configuration difficulties, and inconsistencies in Python packaging related to naming conventions for packages containing hyphens, underscores, or periods. Potential causes discussed include legacy code, lack of time for thorough testing and bug fixing, and economic pressures against developing fully working software systems.
XFLTReaT: A New Dimension in Tunneling (Shakacon 2017)
XFLTReaT presentation from Shakacon 2017
https://www.youtube.com/watch?v=AfqNVXHz0hU
This presentation will sum up how to do tunneling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunneling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunneling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunneling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customized on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunneling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunneling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunneling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Balázs Bucsay - XFLTReaT: Building a Tunnel
XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and offers the capability to the users to take care of only those things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP, RDP or SSH then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 12km on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. It will be show how to tunnel data over a Windows jumpbox utilising RDP (including the dirty low level "secrets") or how to exfiltrate data over ICMP from barely secured networks. We have simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Shall we play a game?
How to run system administrator recruitment process? By creating platform based on open source parts in just 2 nights! I gave this talk in Poland / Kraków OWASP chapter meeting on 17th October 2013 at our local Google for Entrepreneurs site. It's focused on security and also shows how to create recruitment process in CTF / challenge way.
This story covers mostly security details of this whole platform. There's great chance, that I will give another talk about this system but this time focusing on technical details. Stay tuned ;)
JRuby: Pushing the Java Platform Further
The document discusses how JRuby pushes the Java platform further by implementing custom core classes like Array, Hash, String, and IO to match Ruby's behavior exactly. It also describes how JRuby uses libraries like ByteList, Joni, Java Native Runtime, and FFI to provide Ruby-like regular expressions, native I/O, and OS-level features on the JVM. These custom implementations and libraries allow JRuby to overcome challenges like dynamic typing and provide a full-fledged Ruby environment atop the Java Virtual Machine.
Bringing Concurrency to Ruby - RubyConf India 2014
The document discusses bringing concurrency to Ruby. It begins by defining concurrency and parallelism, noting that both are needed but platforms only enable parallelism if jobs can split into concurrent tasks. It reviews concurrency and parallelism in popular Ruby platforms like MRI, JRuby, and Rubinius. The document outlines four rules for concurrency and discusses techniques like immutable data, locking, atomics, and specialized collections for mutable data. It highlights libraries that provide high-level concurrency abstractions like Celluloid for actors and Sidekiq for background jobs.
Breaking SSL using time synchronisation attacks
The document provides an overview of breaking SSL using time synchronization attacks. It discusses how HTTP Strict Transport Security (HSTS) relies on system time being accurate, and how an attacker could exploit vulnerabilities in network time synchronization protocols like NTP to manipulate a victim's system clock and circumvent HSTS protections. It demonstrates this by using a tool called Delorean to spoof NTP time responses and force client systems to incorrect future or past dates. The document also analyzes time synchronization implementations and vulnerabilities on different operating systems.
Perl6 meets JVM
This document discusses using Perl 6 with the Java Virtual Machine (JVM). It provides instructions for installing Perl 6 with JVM support and demonstrates various features like JSON processing, HTTP client/server, database access using SQLite and MySQL, calling Java methods, and building a basic GUI. Performance tests show startup is slow but HTTP server throughput is faster with JVM backend compared to MoarVM backend. Overall, the author finds Perl 6 on JVM to be practical for Java developers needing to access Java libraries and calls it potentially useful.
Trick or XFLTReaT a.k.a. Tunnel All The Things
XFLTReaT presentation from RuxCon 2017
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
XFLTReaT: a new dimension in tunnelling (BruCON 0x09 2017)
XFLTReaT presentation from BruCON 0x09 2017
https://www.youtube.com/watch?v=0hnxgu8lkfc
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Shiny New HTTP Shit
This document summarizes several new and upcoming HTTP specifications and technologies, including:
1. HTTPbis which splits the HTTP specification into seven modular specs to improve understandability.
2. Header i18n which aims to fix internationalization issues in HTTP header fields.
3. Stale cache controls which allow serving stale cached content to hide latency or when problems occur.
4. Prefer which lets clients request an empty response, like for POST/PUT APIs.
EhTrace -- RoP Hooks
This document describes EhTrace, a tool for tracing the execution of binary programs through hooking and branch stepping. EhTrace uses the Windows VEH exception handler to single step programs by setting flags in the CPU context. It can be used to analyze program control flow at the basic block level for purposes like malware analysis, debugging, and code coverage. The document discusses techniques for maintaining control during tracing and fighting attempts by the target program to detect or alter the tracing.
Tuning Linux for MongoDB
This document discusses how to tune Linux for optimal MongoDB performance. Key points include setting ulimits to allow for many processes and open files, disabling transparent huge pages, using the deadline IO scheduler, setting the dirty ratio and swappiness low, and ensuring consistent clocks with NTP. Monitoring tools like Percona PMM or Prometheus with Grafana dashboards can help analyze MongoDB and system metrics.
XFLTReaT: A New Dimension in Tunnelling (HITB GSEC 2017)
XFLTReaT presentation from Hack In The Box GSEC 2017
https://www.youtube.com/watch?v=6EU_RLb2YxI
XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Tips on High Performance Server Programming
This document provides tips and best practices for high performance server programming. It discusses avoiding blocking, using efficient algorithms and data structures, separating I/O from business logic, and tuning for bottlenecks. It also covers various I/O models like blocking, non-blocking, and asynchronous I/O. Key aspects of designing high performance servers include using non-blocking I/O, event-driven architectures, and avoiding excessive threading.
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
This document discusses using distributed locks to synchronize access to shared resources in distributed applications. It provides examples of when distributed locks are needed to prevent data corruption or loss from concurrent requests. The document recommends using proven distributed lock libraries like redis-semaphore and redlock-rb to implement distributed locks instead of recreating the logic. Tips are given on testing for concurrency issues and choosing correctness over efficiency with locks.
Vulnerability, exploit to metasploit
The document discusses developing an exploit from a vulnerability and integrating it into the Metasploit framework. It covers finding a buffer overflow vulnerability in an application called "Free MP3 CD Ripper", using tools like ImmunityDebugger and Mona.py to crash the application and gain control of EIP. It then shows using Mona.py to generate an exploit, testing it works, and submitting it to the Metasploit framework. It also provides an overview of Meterpreter and its capabilities.
High-Performance Networking Using eBPF, XDP, and io_uring
Bryan McCoid discusses using eBPF, XDP, and io_uring for high performance networking. XDP allows programs to process packets in the kernel without loading modules. AF_XDP sockets use eBPF to route packets between kernel and userspace via ring buffers. McCoid is building a Rust runtime called Glommio to interface with these techniques. The runtime integrates with io_uring and allows multiple design patterns for receiving packets from AF_XDP sockets.
Concurrency in Python
It is mainly about the multithreading and the multiprocessing in Python, and *in Python's flavor*.
It's also the share at Taipei.py [1].
[1] http://www.meetup.com/Taipei-py/events/220452029/
Skyscanner presents: Getting started with tornado with Sharknado
Walk through the basics of asynchronous web server programming using tornado. After reviewing the basics of async servers, step by step we’ll build an application that can asynchronously gather quotes from one of the best bad movies of all time. Visit http://medium.com/@skyscannercodevoyagers for more learnings,
Dive into sentry
1. Sentry is an open source error logging and aggregation platform that was originally created at Disqus.
2. It allows developers to capture errors and aggregate them to get a better understanding of issues affecting an application in production.
3. The platform consists of a server component called Sentry that collects and stores errors, and client libraries called Raven that integrate with applications to capture errors and send them to Sentry.
Ruby and Distributed Storage Systems
This document discusses using Ruby for distributed storage systems. It describes components like Bigdam, which is Treasure Data's new data ingestion pipeline. Bigdam uses microservices and a distributed key-value store called Bigdam-pool to buffer data. The document discusses designing and testing Bigdam using mocking, interfaces, and integration tests in Ruby. It also explores porting Bigdam-pool from Java to Ruby and investigating Ruby's suitability for tasks like asynchronous I/O, threading, and serialization/deserialization.
Outside The Box With Apache Cassnadra
Cassandra presentation given at the 3rd annual Palmetto Open Source Software Conference (POSSCON 2010).
Contenu connexe
Tendances
XFLTReaT: A New Dimension in Tunneling (Shakacon 2017)
XFLTReaT presentation from Shakacon 2017
https://www.youtube.com/watch?v=AfqNVXHz0hU
This presentation will sum up how to do tunneling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunneling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunneling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunneling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customized on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunneling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunneling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunneling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Balázs Bucsay - XFLTReaT: Building a Tunnel
XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and offers the capability to the users to take care of only those things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP, RDP or SSH then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 12km on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. It will be show how to tunnel data over a Windows jumpbox utilising RDP (including the dirty low level "secrets") or how to exfiltrate data over ICMP from barely secured networks. We have simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Shall we play a game?
How to run system administrator recruitment process? By creating platform based on open source parts in just 2 nights! I gave this talk in Poland / Kraków OWASP chapter meeting on 17th October 2013 at our local Google for Entrepreneurs site. It's focused on security and also shows how to create recruitment process in CTF / challenge way.
This story covers mostly security details of this whole platform. There's great chance, that I will give another talk about this system but this time focusing on technical details. Stay tuned ;)
JRuby: Pushing the Java Platform Further
The document discusses how JRuby pushes the Java platform further by implementing custom core classes like Array, Hash, String, and IO to match Ruby's behavior exactly. It also describes how JRuby uses libraries like ByteList, Joni, Java Native Runtime, and FFI to provide Ruby-like regular expressions, native I/O, and OS-level features on the JVM. These custom implementations and libraries allow JRuby to overcome challenges like dynamic typing and provide a full-fledged Ruby environment atop the Java Virtual Machine.
Bringing Concurrency to Ruby - RubyConf India 2014
The document discusses bringing concurrency to Ruby. It begins by defining concurrency and parallelism, noting that both are needed but platforms only enable parallelism if jobs can split into concurrent tasks. It reviews concurrency and parallelism in popular Ruby platforms like MRI, JRuby, and Rubinius. The document outlines four rules for concurrency and discusses techniques like immutable data, locking, atomics, and specialized collections for mutable data. It highlights libraries that provide high-level concurrency abstractions like Celluloid for actors and Sidekiq for background jobs.
Breaking SSL using time synchronisation attacks
The document provides an overview of breaking SSL using time synchronization attacks. It discusses how HTTP Strict Transport Security (HSTS) relies on system time being accurate, and how an attacker could exploit vulnerabilities in network time synchronization protocols like NTP to manipulate a victim's system clock and circumvent HSTS protections. It demonstrates this by using a tool called Delorean to spoof NTP time responses and force client systems to incorrect future or past dates. The document also analyzes time synchronization implementations and vulnerabilities on different operating systems.
Perl6 meets JVM
This document discusses using Perl 6 with the Java Virtual Machine (JVM). It provides instructions for installing Perl 6 with JVM support and demonstrates various features like JSON processing, HTTP client/server, database access using SQLite and MySQL, calling Java methods, and building a basic GUI. Performance tests show startup is slow but HTTP server throughput is faster with JVM backend compared to MoarVM backend. Overall, the author finds Perl 6 on JVM to be practical for Java developers needing to access Java libraries and calls it potentially useful.
Trick or XFLTReaT a.k.a. Tunnel All The Things
XFLTReaT presentation from RuxCon 2017
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
XFLTReaT: a new dimension in tunnelling (BruCON 0x09 2017)
XFLTReaT presentation from BruCON 0x09 2017
https://www.youtube.com/watch?v=0hnxgu8lkfc
This presentation will sum up how to do tunnelling with different protocols and will have different perspectives detailed. For example, companies are fighting hard to block exfiltration from their network: they use http(s) proxies, DLP, IPS technologies to protect their data, but are they protected against tunnelling? There are so many interesting questions to answer for users, abusers, companies and malware researchers. Mitigation and bypass techniques will be shown you during this presentation, which can be used to filter any tunnelling on your network or to bypass misconfigured filters.
Our new tool XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Shiny New HTTP Shit
This document summarizes several new and upcoming HTTP specifications and technologies, including:
1. HTTPbis which splits the HTTP specification into seven modular specs to improve understandability.
2. Header i18n which aims to fix internationalization issues in HTTP header fields.
3. Stale cache controls which allow serving stale cached content to hide latency or when problems occur.
4. Prefer which lets clients request an empty response, like for POST/PUT APIs.
EhTrace -- RoP Hooks
This document describes EhTrace, a tool for tracing the execution of binary programs through hooking and branch stepping. EhTrace uses the Windows VEH exception handler to single step programs by setting flags in the CPU context. It can be used to analyze program control flow at the basic block level for purposes like malware analysis, debugging, and code coverage. The document discusses techniques for maintaining control during tracing and fighting attempts by the target program to detect or alter the tracing.
Tuning Linux for MongoDB
This document discusses how to tune Linux for optimal MongoDB performance. Key points include setting ulimits to allow for many processes and open files, disabling transparent huge pages, using the deadline IO scheduler, setting the dirty ratio and swappiness low, and ensuring consistent clocks with NTP. Monitoring tools like Percona PMM or Prometheus with Grafana dashboards can help analyze MongoDB and system metrics.
XFLTReaT: A New Dimension in Tunnelling (HITB GSEC 2017)
XFLTReaT presentation from Hack In The Box GSEC 2017
https://www.youtube.com/watch?v=6EU_RLb2YxI
XFLTReaT is an open-source tunnelling framework that handles all the boring stuff and gives users the capability to take care of only the things that matter. It provides significant improvements over existing tools. From now on there is no need to write a new tunnel for each and every protocol or to deal with interfaces and routing. Any protocol can be converted to a module, which works in a plug-and-play fashion; authentication and encryption can be configured and customised on all traffic and it is also worth mentioning that the framework was designed to be easy to configure, use and develop. In case there is a need to send packets over ICMP type 0 or HTTPS TLS v1.2 with a special header, then this can be done in a matter of minutes, instead of developing a new tool from scratch. The potential use (or abuse) cases are plentiful, such as bypassing network restrictions of an ISP, the proxy of a workplace or obtaining Internet connectivity through bypassing captive portals in the middle of the Atlantic Ocean or at an altitude of 33000ft on an airplane.
This framework is not just a tool; it unites different technologies in the field of tunnelling. While we needed to use different tunnels and VPNs for different protocols in the past like OpenVPN for TCP and UDP, ptunnel for ICMP or iodined for DNS tunnelling, it changes now. After taking a look at these tools it was easy to see some commonality, all of them are doing the same things only the means of communication are different. We simplified the whole process and created a framework that is responsible for everything but the communication itself, we rethought the old way of tunnelling and tried to give something new to the community. After the initial setup the framework takes care of everything. With the check functionality we can even find out, which module can be used on the network, there is no need for any low-level packet fu and hassle. I guarantee that you won’t be disappointed with the tool and the talk, actually you will be richer with an open-source tool.
Tips on High Performance Server Programming
This document provides tips and best practices for high performance server programming. It discusses avoiding blocking, using efficient algorithms and data structures, separating I/O from business logic, and tuning for bottlenecks. It also covers various I/O models like blocking, non-blocking, and asynchronous I/O. Key aspects of designing high performance servers include using non-blocking I/O, event-driven architectures, and avoiding excessive threading.
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
This document discusses using distributed locks to synchronize access to shared resources in distributed applications. It provides examples of when distributed locks are needed to prevent data corruption or loss from concurrent requests. The document recommends using proven distributed lock libraries like redis-semaphore and redlock-rb to implement distributed locks instead of recreating the logic. Tips are given on testing for concurrency issues and choosing correctness over efficiency with locks.
Vulnerability, exploit to metasploit
The document discusses developing an exploit from a vulnerability and integrating it into the Metasploit framework. It covers finding a buffer overflow vulnerability in an application called "Free MP3 CD Ripper", using tools like ImmunityDebugger and Mona.py to crash the application and gain control of EIP. It then shows using Mona.py to generate an exploit, testing it works, and submitting it to the Metasploit framework. It also provides an overview of Meterpreter and its capabilities.
High-Performance Networking Using eBPF, XDP, and io_uring
Bryan McCoid discusses using eBPF, XDP, and io_uring for high performance networking. XDP allows programs to process packets in the kernel without loading modules. AF_XDP sockets use eBPF to route packets between kernel and userspace via ring buffers. McCoid is building a Rust runtime called Glommio to interface with these techniques. The runtime integrates with io_uring and allows multiple design patterns for receiving packets from AF_XDP sockets.
Concurrency in Python
It is mainly about the multithreading and the multiprocessing in Python, and *in Python's flavor*.
It's also the share at Taipei.py [1].
[1] http://www.meetup.com/Taipei-py/events/220452029/
Skyscanner presents: Getting started with tornado with Sharknado
Walk through the basics of asynchronous web server programming using tornado. After reviewing the basics of async servers, step by step we’ll build an application that can asynchronously gather quotes from one of the best bad movies of all time. Visit http://medium.com/@skyscannercodevoyagers for more learnings,
Dive into sentry
1. Sentry is an open source error logging and aggregation platform that was originally created at Disqus.
2. It allows developers to capture errors and aggregate them to get a better understanding of issues affecting an application in production.
3. The platform consists of a server component called Sentry that collects and stores errors, and client libraries called Raven that integrate with applications to capture errors and send them to Sentry.
Tendances (20)
XFLTReaT: A New Dimension in Tunneling (Shakacon 2017)
XFLTReaT: A New Dimension in Tunneling (Shakacon 2017)
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
XFLTReaT: a new dimension in tunnelling (BruCON 0x09 2017)
XFLTReaT: a new dimension in tunnelling (BruCON 0x09 2017)
XFLTReaT: A New Dimension in Tunnelling (HITB GSEC 2017)
XFLTReaT: A New Dimension in Tunnelling (HITB GSEC 2017)
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
High-Performance Networking Using eBPF, XDP, and io_uring
High-Performance Networking Using eBPF, XDP, and io_uring
Skyscanner presents: Getting started with tornado with Sharknado
Skyscanner presents: Getting started with tornado with Sharknado
Similaire à Java for ultra low latency applications
Ruby and Distributed Storage Systems
This document discusses using Ruby for distributed storage systems. It describes components like Bigdam, which is Treasure Data's new data ingestion pipeline. Bigdam uses microservices and a distributed key-value store called Bigdam-pool to buffer data. The document discusses designing and testing Bigdam using mocking, interfaces, and integration tests in Ruby. It also explores porting Bigdam-pool from Java to Ruby and investigating Ruby's suitability for tasks like asynchronous I/O, threading, and serialization/deserialization.
Outside The Box With Apache Cassnadra
Cassandra presentation given at the 3rd annual Palmetto Open Source Software Conference (POSSCON 2010).
Ehcache3 — JSR-107 on steroids
Presented at JavaOne 2015.
JSR107, aka the Temporary Caching API for the Java Platform, has now been finalized almost 2 years ago. We've heard all about its ease of use and capabilities. But there is much left unaddressed. The good news is that the EG is looking at addressing many of the current shortcomings... But what do you do now? Go for proprietary APIs?!
Ehcache, the de facto caching API for 10 years now, has gone through a major API revamp: Ehcache3. One major theme, beyond its usual ease of use, was JSR107. Natively integrating it, but also looking beyond. With close to no API tie-ins, Ehcache3 lets you extend the JSR107 API transparently to go beyond the specification: topology-wise: whether you want to go offheap and scale up, or scale out by clustering your caches; functionality-wise: using transactional caches, automatic resource control or even using a write-behind cache to scale out writes...
Best of all is that this isn't only minimally intrusive, it is also all free to use and available as part of the open-source Ehcache v3 that has been GA'ed earlier this year...
The Cassandra Distributed Database
This document summarizes Cassandra, an open source distributed database management system designed to handle large amounts of data across many commodity servers. It discusses Cassandra's history, key features like tunable consistency levels and support for structured and indexed columns. Case studies describe how companies like Digg, Twitter, Facebook and Mahalo use Cassandra to handle terabytes of data and high transaction volumes. The roadmap outlines upcoming releases that will improve features like compaction, management tools, and support for dynamic schema changes.
Ehcache 3 @ BruJUG
- The document provides an overview of Ehcache 3, a caching framework that is a major upgrade from Ehcache 2. It highlights Ehcache 3's type safe APIs, fluent configuration, support for multiple storage tiers including heap, off-heap and disk, and fully compliant support for JSR-107 caching standard. It also discusses some of Ehcache 3's new features like expiry, eviction advisors, and cache loaders/writers, as well as features that were dropped like search and explicit locking. The presentation aims to explain Ehcache 3's motivation and significant changes from Ehcache 2.
Hacking Blind
We show that it is possible to write remote stack buffer overflow exploits without possessing a copy of the target binary or source code, against services that restart after a crash. This makes it possible to hack proprietary closed-binary services, or open-source servers manually compiled and installed from source where the binary remains unknown to the attacker. Traditional techniques are usually paired against a particular binary and distribution where the hacker knows the location of useful gadgets for Return Oriented Programming (ROP). Our Blind ROP (BROP) attack instead remotely finds enough ROP gadgets to perform a write system call and transfers the vulnerable binary over the network, after which an exploit can be completed using known techniques. This is accomplished by leaking a single bit of information based on whether a process crashed or not when given a particular input string. BROP requires a stack vulnerability and a service that restarts after a crash. The attack works against modern 64-bit Linux with address space layout randomization (ASLR), no-execute page protection (NX) and stack canaries.
Hacking blind
We show that it is possible to write remote stack buffer overflow exploits without possessing a copy of the target binary or source code, against services that restart after a crash. This makes it possible to hack proprietary closed-binary services, or open-source servers manually compiled and installed from source where the binary remains unknown to the attacker. Traditional techniques are usually paired against a particular binary and distribution where the hacker knows the location of useful gadgets for Return Oriented Programming (ROP). Our Blind ROP (BROP) attack instead remotely finds enough ROP gadgets to perform a write system call and transfers the vulnerable binary over the network, after which an exploit can be completed using known techniques. This is accomplished by leaking a single bit of information based on whether a process crashed or not when given a particular input string. BROP requires a stack vulnerability and a service that restarts after a crash. The attack works against modern 64-bit Linux with address space layout randomization (ASLR), no-execute page protection (NX) and stack canaries.
Making Scala Faster: 3 Expert Tips For Busy Development Teams
This document provides information about the author and discusses ways to improve Scala compilation performance. The author has worked on Scala tooling and is the co-founder of Triplequote. They discuss how build time differs from compilation time due to pre-processing steps. They warn that type classes and macros can significantly increase code size and slow compilation. Whitebox macros are type-checked three times while blackbox macros only participate in type inference. They recommend monitoring compilation to identify bottlenecks like macro expansion or implicit resolution. Finally, they note that the Scala compiler is single-threaded, but parallelization using Scala Hydra can improve compilation speed.
Scalability, Availability & Stability Patterns
This document provides an overview of patterns for scalability, availability, and stability in distributed systems. It discusses general recommendations like immutability and referential transparency. It covers scalability trade-offs around performance vs scalability, latency vs throughput, and availability vs consistency. It then describes various patterns for scalability including managing state through partitioning, caching, sharding databases, and using distributed caching. It also covers patterns for managing behavior through event-driven architecture, compute grids, load balancing, and parallel computing. Availability patterns like fail-over, replication, and fault tolerance are discussed. The document provides examples of popular technologies that implement many of these patterns.
Introduction to Storm
Storm is a distributed and fault-tolerant realtime computation system. It was created at BackType/Twitter to analyze tweets, links, and users on Twitter in realtime. Storm provides scalability, reliability, and ease of programming. It uses components like Zookeeper, ØMQ, and Thrift. A Storm topology defines the flow of data between spouts that read data and bolts that process data. Storm guarantees processing of all data through its reliability APIs and guarantees no data loss even during failures.
Follow the White Rabbit - Message Queues with PHP
An introduction to message queues with PHP. We'll focus on RabbitMQ and how to leverage queuing scenarios in your applications. The talk will cover the main concepts of RabbitMQ server and AMQP protocol and show how to use it in PHP. The RabbitMqBundle for Symfony2 will be presented and we'll see how easy you can start to use message queuing in minutes.
Presented at Symfony User Group Belgium: http://www.meetup.com/Symfony-User-Group-Belgium/events/169953362/
Search at Twitter: Presented by Michael Busch, Twitter
Twitter processes over 500 million tweets per day and more than 2 billion search queries per day. The company uses a search architecture based on Lucene with custom extensions. This includes an in-memory real-time index optimized for concurrency without locks, and a schema-based document factory. Future work includes support for parallel index segments and additional Lucene features.
Introduction to Apache NiFi And Storm
This slide shares an introduction to Apache NiFi and Storm. For Apache Storm the slide also talks about future of Apache Storm quickly.
Invoke-CradleCrafter: Moar PowerShell obFUsk8tion & Detection (@('Tech','niqu...
These are the slides for my presentation at x33fcon (April 28, 2017) where I publicly released Invoke-CradleCrafter.
The Economies of Scaling Software
This document discusses quantifying the scalability of software. It recommends instrumenting code from the beginning to collect monitoring data on application health, the entire cluster, and individual nodes' system resources. This allows measuring how well a system can handle increasing load and evolving constraints.
Past, Present, and Future of Apache Storm
This document discusses the past, present, and future of Apache Storm. It summarizes key releases of Storm including 0.9.x which moved Storm to Apache, 0.10.x which focused on enterprise readiness, 1.0 which improved performance and maturity, and 1.1.0 which added features like streaming SQL. It outlines current features such as the Resource Aware Scheduler, Kafka integration improvements, and new integrations. It concludes by discussing Storm 2.0 and beyond, including transitions to Java from Clojure and improved performance, as well as integrating Apache Beam and allowing bounded spouts and dynamic topology updates.
Apache storm vs. Spark Streaming
Slides for an upcoming talk about Apache Storm and Spark Streaming.
This is a draft and is subject to change. Comments welcome.
Alluxio - Scalable Filesystem Metadata Services
This talk was presented by Alluxio's top contributor and PMC Maintainer Calvin Jia at the Alluxio bay area Meetup.
This talk shares our design, implementation and optimization of Alluxio metadata service to address the scalability challenges, focusing on how to apply and combine techniques including tiered metadata storage (based on off-heap KV store RocksDB), fine-grained file system inode tree locking scheme, embedded state-replicate machine (based on RAFT), exploration and performance tuning in the correct RPC frameworks (thrift vs gRPC) and etc.
The economies of scaling software - Abdel Remani
This document discusses quantifying the scalability of software. It recommends instrumenting code from the beginning to collect monitoring data on application health, the entire cluster, and individual nodes' system resources. This allows measuring how well a system can handle increasing load and evolving constraints.
High Performance Systems in Go - GopherCon 2014
Short presentation on some techniques to gain performance of the NATS messaging system as it was rewritten in Go.
Similaire à Java for ultra low latency applications (20)
Making Scala Faster: 3 Expert Tips For Busy Development Teams
Making Scala Faster: 3 Expert Tips For Busy Development Teams
Search at Twitter: Presented by Michael Busch, Twitter
Search at Twitter: Presented by Michael Busch, Twitter
Invoke-CradleCrafter: Moar PowerShell obFUsk8tion & Detection (@('Tech','niqu...
Invoke-CradleCrafter: Moar PowerShell obFUsk8tion & Detection (@('Tech','niqu...
Plus de Rafael Monteiro e Pereira
Kubeflow - a caixa de ferramentas para tirar seus modelos do notebook e coloc...
Kubeflow - a caixa de ferramentas para tirar seus modelos do notebook e coloc...Rafael Monteiro e Pereira
Kubeflow was created to help data science teams deploy machine learning models into production by providing tools to train and serve models on Kubernetes clusters. Currently, many data science teams use laptops and local machines to develop models using open source tools and only share results via presentations instead of deploying APIs. When a model is approved, it is handed to software engineers to rewrite in a production language, which can change the model and takes significant time. Kubeflow aims to help data scientists deploy models directly without rewriting by providing tools to manage end-to-end ML workflows on Kubernetes.Microservices: Patterns & Practices
This document summarizes a presentation on microservices patterns and practices. It introduces microservices architecture and its benefits, such as enabling continuous delivery, improved fault isolation, and eliminating long-term commitment to technology stacks. It then covers several data patterns for microservices including shared database, one database per service, CQRS, and event sourcing. Integration patterns like service discovery, circuit breakers, API gateways, and sagas are also discussed. The document concludes by exploring testing, operations, and deployment patterns for microservices.
Inside the JVM - Performance & Garbage Collector Tuning in JAVA
Sua aplicação deu crash ? Consumindo muita memória ? Lentidão ? Vamos falar sobre como a JVM funciona, como coletar métricas e realizar o tuning na performance de aplicações utilizando as ferramentas nativas da JVM. Além de detecção e correção de problemas como memory leaks ou freezing causado pelo Garbage Collector
Do mainframe para java, microservices e big data
Slides of my Qcon presentation about Offloading data from the mainframe using Kafka, Microservices and Cassandra.
https://qconsp.com/sp2018/presentation/evoluindo-arquitetura-do-itau
https://qconsp.com/sp2018/presentation/evoluindo-arquitetura-do-itau
Cassandra o que nao te contaram
Rafael Monteiro discusses common mistakes made when using Cassandra as an infrastructure for databases. Some key mistakes include sharing storage across nodes, putting the commit log and data on the same disks, and putting a load balancer between the client and cluster. Other design mistakes include using Cassandra as a relational database when the data is not denormalized, having too many materialized views, and filtering by columns that are not in the primary key. Proper data modeling and running nodetool repair can help avoid issues. The presentation encourages using the right tool for the job.
Explorando protocolos de serialização de alto desempenho
O documento discute protocolos de serialização de alto desempenho, comparando o desempenho de serialização e desserialização de mensagens usando protocolos como Protocol Buffers, FlatBuffers e JSON. Os resultados do benchmark mostram que Protocol Buffers e FlatBuffers proporcionam desempenho superior ao JSON, enquanto FlatBuffers é o mais rápido para desserialização de uma única mensagem.
Plus de Rafael Monteiro e Pereira (6)
Kubeflow - a caixa de ferramentas para tirar seus modelos do notebook e coloc...
Kubeflow - a caixa de ferramentas para tirar seus modelos do notebook e coloc...
Inside the JVM - Performance & Garbage Collector Tuning in JAVA
Inside the JVM - Performance & Garbage Collector Tuning in JAVA
Explorando protocolos de serialização de alto desempenho
Explorando protocolos de serialização de alto desempenho
Dernier
UI5con 2024 - Bring Your Own Design System
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
Empowering Growth with Best Software Development Company in Noida - Deuglo
Do you want Software for your Business? Visit Deuglo
Deuglo has top Software Developers in India. They are experts in software development and help design and create custom Software solutions.
Deuglo follows seven steps methods for delivering their services to their customers. They called it the Software development life cycle process (SDLC).
Requirement — Collecting the Requirements is the first Phase in the SSLC process.
Feasibility Study — after completing the requirement process they move to the design phase.
Design — in this phase, they start designing the software.
Coding — when designing is completed, the developers start coding for the software.
Testing — in this phase when the coding of the software is done the testing team will start testing.
Installation — after completion of testing, the application opens to the live server and launches!
Maintenance — after completing the software development, customers start using the software.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
Vitthal Shirke Java Microservices Resume.pdf
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Using Xen Hypervisor for Functional Safety
An update on making Xen hypervisor functionally safe and enhancing its usage in automotive and industrial use cases
Oracle 23c New Features For DBAs and Developers.pptx
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
Zoom is a comprehensive platform designed to connect individuals and teams efficiently. With its user-friendly interface and powerful features, Zoom has become a go-to solution for virtual communication and collaboration. It offers a range of tools, including virtual meetings, team chat, VoIP phone systems, online whiteboards, and AI companions, to streamline workflows and enhance productivity.
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
Microservice Teams - How the cloud changes the way we work
A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams?
Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
How to write a program in any programming language
How to write a program in any programming language
Using Query Store in Azure PostgreSQL to Understand Query Performance
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Webinar On-Demand: Using Flutter for Embedded
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Presentation from Ghazal Aakel and Eric Jochum at the GSD Community Stage Meetup on June 6th, 2024
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
When it is all about ERP solutions, companies typically meet their needs with common ERP solutions like SAP, Oracle, and Microsoft Dynamics. These big players have demonstrated that ERP systems can be either simple or highly comprehensive. This remains true today, but there are new factors to consider, including a promising new contender in the market that’s Odoo. This blog compares Odoo ERP with traditional ERP systems and explains why many companies now see Odoo ERP as the best choice.
What are ERP Systems?
An ERP, or Enterprise Resource Planning, system provides your company with valuable information to help you make better decisions and boost your ROI. You should choose an ERP system based on your company’s specific needs. For instance, if you run a manufacturing or retail business, you will need an ERP system that efficiently manages inventory. A consulting firm, on the other hand, would benefit from an ERP system that enhances daily operations. Similarly, eCommerce stores would select an ERP system tailored to their needs.
Because different businesses have different requirements, ERP system functionalities can vary. Among the various ERP systems available, Odoo ERP is considered one of the best in the ERp market with more than 12 million global users today.
Odoo is an open-source ERP system initially designed for small to medium-sized businesses but now suitable for a wide range of companies. Odoo offers a scalable and configurable point-of-sale management solution and allows you to create customised modules for specific industries. Odoo is gaining more popularity because it is built in a way that allows easy customisation, has a user-friendly interface, and is affordable. Here, you will cover the main differences and get to know why Odoo is gaining attention despite the many other ERP systems available in the market.
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech is a premier mobile app development company in Noida, providing cutting-edge solutions for businesses.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
Dernier (20)
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
How to write a program in any programming language
How to write a program in any programming language
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Java for ultra low latency applications
- 1. Java for ultra low latency applications Rafael Monteiro e Pereira Software Engineer
- 2. What is Latency ? • From the Queueing Theory
- 3. What is Latency ? • From the Queueing Theory
- 4. - Thread Communication - Memory Management
- 5. Thread Communication • Just Create a Queue right ? • You are sure that you wanna block ? • https://github.com/real-logic/benchmarks
- 6. Thread Communication • Burst (100 messages) LinkedBlockingQueue 154 μs ArrayBlockingQueue 56 μs ConcurrentLinkedQueue 44 μs Disruptor 3.0 20 μs
- 7. Thread Communication • Disruptor (https://github.com/LMAX-Exchange/disruptor ) • Very fast Inter-Thread Message Library Source: https://martinfowler.com/articles/images/lmax/arch-full.png
- 10. Off-Heap Allocation • Allocates memory outside the Heap • HARD ! Really !
- 11. Off-Heap Allocation • Copy Object outside the Heap
- 12. Off-Heap Allocation • Use a LIB ! • Open HFT -> https://github.com/OpenHFT • https://github.com/OpenHFT/Java-Thread-Affinity • https://github.com/OpenHFT/Chronicle-Queue • https://github.com/OpenHFT/Chronicle-Logger • Some Tips • Be careful with big Heap sizes. Usually is not a good idea to go beyond 16 GB on Hotspot. • Big Heap == Slow GC !!! Consider using off-heap allocation if a lot of information must be maintained in memory.
- 13. Obrigado!!!! Feedback ? Contato ? E-mail: rafael@monteiro.io Github: github.com/rafamonteiro Twitter: @monteiro_io LinkedIn: lnked.in/rafamonteiro