The document discusses Terracotta's OffHeap library, which provides a map implementation that stores data outside the Java heap in direct memory to avoid garbage collection overhead. It summarizes how OffHeap maps replace standard JDK maps by storing keys and values directly in native memory rather than as objects. It also describes how OffHeap handles serialization, concurrency, and other aspects to emulate concurrent maps without using the Java heap.
The document discusses caching and the JSR-107 caching API. It provides an agenda for topics to be covered, including why caches are used, the JSR-107 API, caching in applications using cache-aside and cache-through approaches, and resilience strategies. It then demonstrates caching code examples and discusses some of the nuances of using JSR-107 for cache-through implementations.
Ehcache 3: JSR-107 on steroids at Devoxx MoroccoLouis Jacomet
The document summarizes Louis Jacomet's presentation on Ehcache 3. Some key points:
- Ehcache 3 is based on JSR-107 and improves on caching APIs and semantics. It supports explicit lifecycles, proper typing for caches, and custom serialization.
- Ehcache 3 enables cache-through and write-behind patterns to integrate caching with databases. It allows caching and persisting data across multiple tiers including heap, disk, and clustered storage.
- New features in Ehcache 3 include eviction veto/prioritization, transactions, automatic resource control, and resilience strategies to handle storage and timeout failures. Clustering is implemented using Terracotta.
- The document provides an overview of Ehcache 3, a caching framework that is a major upgrade from Ehcache 2. It highlights Ehcache 3's type safe APIs, fluent configuration, support for multiple storage tiers including heap, off-heap and disk, and fully compliant support for JSR-107 caching standard. It also discusses some of Ehcache 3's new features like expiry, eviction advisors, and cache loaders/writers, as well as features that were dropped like search and explicit locking. The presentation aims to explain Ehcache 3's motivation and significant changes from Ehcache 2.
This presentation explains how to deploy and use the Integrated Caching feature on Netscaler. I gave this presentation to Citrix staff, customers and partners in worldwide in 2011. The presentation covers best practices and gotchas :) Integrated Caching is an excellent feature that can greatly improve the performance of your website.
This is Apache ZooKeeper session.
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.
By the end of this presentation you should be fairly clear about Apache ZooKeeper.
To watch the video or know more about the course, please visit
http://www.knowbigdata.com/page/big-data-and-hadoop-online-instructor-led-training
Introduction to Apache ZooKeeper | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2kvXlPd
This CloudxLab Introduction to Apache ZooKeeper tutorial helps you to understand ZooKeeper in detail. Below are the topics covered in this tutorial:
1) Data Model
2) Znode Types
3) Persistent Znode
4) Sequential Znode
5) Architecture
6) Election & Majority Demo
7) Why Do We Need Majority?
8) Guarantees - Sequential consistency, Atomicity, Single system image, Durability, Timeliness
9) ZooKeeper APIs
10) Watches & Triggers
11) ACLs - Access Control Lists
12) Usecases
13) When Not to Use ZooKeeper
The document discusses caching and the JSR-107 caching API. It provides an agenda for topics to be covered, including why caches are used, the JSR-107 API, caching in applications using cache-aside and cache-through approaches, and resilience strategies. It then demonstrates caching code examples and discusses some of the nuances of using JSR-107 for cache-through implementations.
Ehcache 3: JSR-107 on steroids at Devoxx MoroccoLouis Jacomet
The document summarizes Louis Jacomet's presentation on Ehcache 3. Some key points:
- Ehcache 3 is based on JSR-107 and improves on caching APIs and semantics. It supports explicit lifecycles, proper typing for caches, and custom serialization.
- Ehcache 3 enables cache-through and write-behind patterns to integrate caching with databases. It allows caching and persisting data across multiple tiers including heap, disk, and clustered storage.
- New features in Ehcache 3 include eviction veto/prioritization, transactions, automatic resource control, and resilience strategies to handle storage and timeout failures. Clustering is implemented using Terracotta.
- The document provides an overview of Ehcache 3, a caching framework that is a major upgrade from Ehcache 2. It highlights Ehcache 3's type safe APIs, fluent configuration, support for multiple storage tiers including heap, off-heap and disk, and fully compliant support for JSR-107 caching standard. It also discusses some of Ehcache 3's new features like expiry, eviction advisors, and cache loaders/writers, as well as features that were dropped like search and explicit locking. The presentation aims to explain Ehcache 3's motivation and significant changes from Ehcache 2.
This presentation explains how to deploy and use the Integrated Caching feature on Netscaler. I gave this presentation to Citrix staff, customers and partners in worldwide in 2011. The presentation covers best practices and gotchas :) Integrated Caching is an excellent feature that can greatly improve the performance of your website.
This is Apache ZooKeeper session.
ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.
By the end of this presentation you should be fairly clear about Apache ZooKeeper.
To watch the video or know more about the course, please visit
http://www.knowbigdata.com/page/big-data-and-hadoop-online-instructor-led-training
Introduction to Apache ZooKeeper | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2kvXlPd
This CloudxLab Introduction to Apache ZooKeeper tutorial helps you to understand ZooKeeper in detail. Below are the topics covered in this tutorial:
1) Data Model
2) Znode Types
3) Persistent Znode
4) Sequential Znode
5) Architecture
6) Election & Majority Demo
7) Why Do We Need Majority?
8) Guarantees - Sequential consistency, Atomicity, Single system image, Durability, Timeliness
9) ZooKeeper APIs
10) Watches & Triggers
11) ACLs - Access Control Lists
12) Usecases
13) When Not to Use ZooKeeper
Running High Performance & Fault-tolerant Elasticsearch Clusters on DockerSematext Group, Inc.
This document discusses running Elasticsearch clusters on Docker containers. It describes how Docker containers are more lightweight than virtual machines and have less overhead. It provides examples of running official Elasticsearch Docker images and customizing configurations. It also covers best practices for networking, storage, constraints, and high availability when running Elasticsearch on Docker.
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTanel Poder
The document describes troubleshooting a performance issue involving parallel data loads into a data warehouse. It is determined that the slowness is due to recursive locking and buffer busy waits occurring during inserts into the SEG$ table as new segments are created by parallel CREATE TABLE AS SELECT statements. This is causing a nested locking ping-pong effect between the cache, transaction, and I/O layers as sessions repeatedly acquire and release locks and buffers.
In Memory Database In Action by Tanel Poder and Kerry OsborneEnkitec
The document discusses Oracle Database In-Memory option and how it improves performance of data retrieval and processing queries. It provides examples of running a simple aggregation query with and without various performance features like In-Memory, vector processing and bloom filters enabled. Enabling these features reduces query elapsed time from 17 seconds to just 3 seconds by minimizing disk I/O and leveraging CPU optimizations like SIMD vector processing.
Oracle Latch and Mutex Contention TroubleshootingTanel Poder
This is an intro to latch & mutex contention troubleshooting which I've delivered at Hotsos Symposium, UKOUG Conference etc... It's also the starting point of my Latch & Mutex contention sections in my Advanced Oracle Troubleshooting online seminar - but we go much deeper there :-)
Use case for using the ElastiCache for Redis in production知教 本間
This document discusses using ElastiCache for Redis in production. It begins with an introduction to the author and their role at gumi Inc. It then provides an overview of Redis operations and data persistence methods. The document outlines gumi's migration from memcached to Redis, including their use of master-slave replication and multiple databases. It notes potential issues with ElastiCache for Redis like lack of auto failover and inability to change instance types. In summary, while some features are still missing, ElastiCache for Redis allows quick node provisioning and has proven reliable for gumi despite initial function gaps.
Empowering developers to deploy their own data storesTomas Doran
Empowering developers to deploy their own data stores using Terrafom, Puppet and rage. A talk about automating server building and configuration for Elasticsearch clusters, using Hashicorp and puppet labs tool. Presented at Config Management Camp 2016 in Ghent
Data Analytics Service Company and Its Ruby UsageSATOSHI TAGOMORI
Treasure Data is a data analytics service company that makes heavy use of Ruby in its platform and services. It uses Ruby for components like Fluentd (log collection), Embulk (data loading), scheduling, and its Rails-based API and console. Java and JRuby are also used for components involving Hadoop and Presto processing. The company's architecture includes collectors that ingest data, a PlazmaDB for storage, workers that process jobs on Hadoop and Presto clusters, and schedulers that queue and schedule those jobs using technologies like PerfectSched and PerfectQueue which are written in Ruby. Hive jobs are built programmatically using Ruby to generate configurations and submit the jobs to underlying Hadoop clusters.
Description of some of the elements that go in to creating a PostgreSQL-as-a-Service for organizations with many teams and a diverse ecosystem of applications and teams.
The document discusses asynchronous and non-blocking I/O with JRuby. It explains that asynchronous operations are better than synchronous operations because they use fewer resources and allow for parallelism. It provides an example of building a JRuby application with the Ratpack framework that makes asynchronous HTTP requests to eBay's API in a non-blocking way using promises. It also discusses using RxJava and Hystrix with Ratpack to build a book management application that handles data and API requests asynchronously.
Today's high-traffic web sites must implement performance-boosting measures that reduce data processing and reduce load on the database, while increasing the speed of content delivery. One such method is the use of a cache to temporarily store whole pages, database recordsets, large objects, and sessions. While many caching mechanisms exist, memcached provides one of the fastest and easiest-to-use caching servers. Coupling memcached with the alternative PHP cache (APC) can greatly improve performance by reducing data processing time. In this talk, Ben Ramsey covers memcached and the pecl/memcached and pecl/apc extensions for PHP, exploring caching strategies, a variety of configuration options to fine-tune your caching solution, and discusses when it may be appropriate to use memcached vs. APC to cache objects or data.
Regex Considered Harmful: Use Rosie Pattern Language InsteadAll Things Open
The document discusses using the Rosie Pattern Language (RPL) instead of regular expressions for parsing log and data files. RPL aims to address issues with regex like readability, maintainability, and performance. It describes how RPL is designed like a programming language with common patterns. RPL patterns are loaded into the Rosie Pattern Engine which can parse files and annotate text with semantic tags.
The document discusses continuous deployment and practices at Disqus for releasing code frequently. It emphasizes shipping code as soon as it is ready after it has been reviewed, passes automated tests, and some level of QA. It also discusses keeping development simple, integrating code changes through automated testing, using metrics for reporting, and doing progressive rollouts of new features to subsets of users.
5 Takeaways from Migrating a Library to Scala 3 - Scala LoveNatan Silnitsky
Scala 3 is going to make Scala more easy to write, and especially read. more power features like enums and less misleading keywords like implicit.
But first we need to migrate our old Scala 2.12 / 2.13 codebase to Scala 3.
This talk tells the story of how I tried to migrate greyhound open-source library to Scala 3 with partial success.
You will hear about what works, what doesn't and about a few pitfalls to avoid
Migration takeaways include:
1. use migration tools, don't do it manually
2. which popular 3rd party libraries can and can't be used by Scala 3 code
and many more
HBaseConEast2016: Practical Kerberos with Apache HBaseMichael Stack
- The document is a slide presentation on practical Kerberos with Apache HBase given by Josh Elser of Hortonworks.
- It provides an introduction to Kerberos, how it is used for authentication in HBase and Hadoop, and best practices for configuration and troubleshooting common issues.
- Key aspects covered include how Kerberos tickets and keytabs are used, the SASL and GSSAPI protocols that enable authenticated RPC, and approaches like delegation tokens and proxy users that handle special cases like long-running jobs.
JRuby allows developers to write plugins for data processing systems like Norikra and Embulk in Ruby while taking advantage of Java libraries and the JVM. Norikra is a stream processing system that allows SQL queries over data streams. It is written in JRuby and uses the Java Esper library. Embulk is an open-source ETL tool that loads data between databases and file formats using plugins. Both systems use a plugin architecture where plugins can be written in JRuby or Java and are distributed as Ruby gems. This allows for a pluggable ecosystem that benefits from Ruby's productivity while utilizing Java libraries and the JVM's performance.
Logging for Production Systems in The Container Era discusses how to effectively collect and analyze logs and metrics in microservices-based container environments. It introduces Fluentd as a centralized log collection service that supports pluggable input/output, buffering, and aggregation. Fluentd allows collecting logs from containers and routing them to storage systems like Kafka, HDFS and Elasticsearch. It also supports parsing, filtering and enriching log data through plugins.
Memcached is a free and open-source distributed memory caching system that can be used to speed up dynamic web applications by reducing database load. It stores objects in memory to return frequently or recently used results very quickly. Common things to cache include query results, objects with heavy calculations, and anything that takes time to generate like database calls, API calls, or page rendering. The memcached client knows all memcached servers and hashes keys to determine which server to store or retrieve each object from. Objects are stored using keys and have a maximum size of 1MB. Commands like get, set, add, delete are used to interact with the cache.
This document provides instructions for setting up Apache Kafka and Spark Streaming to process streaming data from Kafka with Spark. It describes how to install Zookeeper and Kafka, create a Kafka topic, produce and consume messages, and run the KafkaWordCount Spark Streaming example application to perform word count on the streaming data from Kafka. It also explains the different processing semantics supported by Spark Streaming for Kafka integration.
Fast as C: How to Write Really Terrible JavaCharles Nutter
For years we’ve been told that the JVM’s amazing optimizers can take your running code and make it “fast” or “as fast as C++” or “as fast as C”…or sometimes “faster than C”. And yet we don’t often see this happen in practice, due in large part to (good and bad) development patterns that have taken hold in the Java world.
In this talk, we’ll explore the main reasons why Java code rarely runs as fast as C or C++ and how you can write really bad Java code that the JVM will do a better job of optimizing. We’ll take some popular microbenchmarks and burn them to the ground, monitoring JIT logs and assembly dumps along the way.
PostgreSQL as seen by Rubyists (Kaigi on Rails 2022)Андрей Новиков
PostgreSQL has become the most popular RDBMS in the Ruby ecosystem in the last decade. It has a great set of built-in features, including a variety of versatile data types, both common and very specific.
But when we load data from the database to our application code, we're working with Ruby data types: classes from the standard library, Rails, or other gems. So while they can seem to be the same as their PostgreSQL counterparts, they are not absolutely identical, and sometimes that could lead to surprising behavior.
In this talk, I would like to explore the power of data types in PostgreSQL and Ruby and how to work with them properly to use both Ruby and PostgreSQL on 100% of their power!
Running High Performance & Fault-tolerant Elasticsearch Clusters on DockerSematext Group, Inc.
This document discusses running Elasticsearch clusters on Docker containers. It describes how Docker containers are more lightweight than virtual machines and have less overhead. It provides examples of running official Elasticsearch Docker images and customizing configurations. It also covers best practices for networking, storage, constraints, and high availability when running Elasticsearch on Docker.
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTanel Poder
The document describes troubleshooting a performance issue involving parallel data loads into a data warehouse. It is determined that the slowness is due to recursive locking and buffer busy waits occurring during inserts into the SEG$ table as new segments are created by parallel CREATE TABLE AS SELECT statements. This is causing a nested locking ping-pong effect between the cache, transaction, and I/O layers as sessions repeatedly acquire and release locks and buffers.
In Memory Database In Action by Tanel Poder and Kerry OsborneEnkitec
The document discusses Oracle Database In-Memory option and how it improves performance of data retrieval and processing queries. It provides examples of running a simple aggregation query with and without various performance features like In-Memory, vector processing and bloom filters enabled. Enabling these features reduces query elapsed time from 17 seconds to just 3 seconds by minimizing disk I/O and leveraging CPU optimizations like SIMD vector processing.
Oracle Latch and Mutex Contention TroubleshootingTanel Poder
This is an intro to latch & mutex contention troubleshooting which I've delivered at Hotsos Symposium, UKOUG Conference etc... It's also the starting point of my Latch & Mutex contention sections in my Advanced Oracle Troubleshooting online seminar - but we go much deeper there :-)
Use case for using the ElastiCache for Redis in production知教 本間
This document discusses using ElastiCache for Redis in production. It begins with an introduction to the author and their role at gumi Inc. It then provides an overview of Redis operations and data persistence methods. The document outlines gumi's migration from memcached to Redis, including their use of master-slave replication and multiple databases. It notes potential issues with ElastiCache for Redis like lack of auto failover and inability to change instance types. In summary, while some features are still missing, ElastiCache for Redis allows quick node provisioning and has proven reliable for gumi despite initial function gaps.
Empowering developers to deploy their own data storesTomas Doran
Empowering developers to deploy their own data stores using Terrafom, Puppet and rage. A talk about automating server building and configuration for Elasticsearch clusters, using Hashicorp and puppet labs tool. Presented at Config Management Camp 2016 in Ghent
Data Analytics Service Company and Its Ruby UsageSATOSHI TAGOMORI
Treasure Data is a data analytics service company that makes heavy use of Ruby in its platform and services. It uses Ruby for components like Fluentd (log collection), Embulk (data loading), scheduling, and its Rails-based API and console. Java and JRuby are also used for components involving Hadoop and Presto processing. The company's architecture includes collectors that ingest data, a PlazmaDB for storage, workers that process jobs on Hadoop and Presto clusters, and schedulers that queue and schedule those jobs using technologies like PerfectSched and PerfectQueue which are written in Ruby. Hive jobs are built programmatically using Ruby to generate configurations and submit the jobs to underlying Hadoop clusters.
Description of some of the elements that go in to creating a PostgreSQL-as-a-Service for organizations with many teams and a diverse ecosystem of applications and teams.
The document discusses asynchronous and non-blocking I/O with JRuby. It explains that asynchronous operations are better than synchronous operations because they use fewer resources and allow for parallelism. It provides an example of building a JRuby application with the Ratpack framework that makes asynchronous HTTP requests to eBay's API in a non-blocking way using promises. It also discusses using RxJava and Hystrix with Ratpack to build a book management application that handles data and API requests asynchronously.
Today's high-traffic web sites must implement performance-boosting measures that reduce data processing and reduce load on the database, while increasing the speed of content delivery. One such method is the use of a cache to temporarily store whole pages, database recordsets, large objects, and sessions. While many caching mechanisms exist, memcached provides one of the fastest and easiest-to-use caching servers. Coupling memcached with the alternative PHP cache (APC) can greatly improve performance by reducing data processing time. In this talk, Ben Ramsey covers memcached and the pecl/memcached and pecl/apc extensions for PHP, exploring caching strategies, a variety of configuration options to fine-tune your caching solution, and discusses when it may be appropriate to use memcached vs. APC to cache objects or data.
Regex Considered Harmful: Use Rosie Pattern Language InsteadAll Things Open
The document discusses using the Rosie Pattern Language (RPL) instead of regular expressions for parsing log and data files. RPL aims to address issues with regex like readability, maintainability, and performance. It describes how RPL is designed like a programming language with common patterns. RPL patterns are loaded into the Rosie Pattern Engine which can parse files and annotate text with semantic tags.
The document discusses continuous deployment and practices at Disqus for releasing code frequently. It emphasizes shipping code as soon as it is ready after it has been reviewed, passes automated tests, and some level of QA. It also discusses keeping development simple, integrating code changes through automated testing, using metrics for reporting, and doing progressive rollouts of new features to subsets of users.
5 Takeaways from Migrating a Library to Scala 3 - Scala LoveNatan Silnitsky
Scala 3 is going to make Scala more easy to write, and especially read. more power features like enums and less misleading keywords like implicit.
But first we need to migrate our old Scala 2.12 / 2.13 codebase to Scala 3.
This talk tells the story of how I tried to migrate greyhound open-source library to Scala 3 with partial success.
You will hear about what works, what doesn't and about a few pitfalls to avoid
Migration takeaways include:
1. use migration tools, don't do it manually
2. which popular 3rd party libraries can and can't be used by Scala 3 code
and many more
HBaseConEast2016: Practical Kerberos with Apache HBaseMichael Stack
- The document is a slide presentation on practical Kerberos with Apache HBase given by Josh Elser of Hortonworks.
- It provides an introduction to Kerberos, how it is used for authentication in HBase and Hadoop, and best practices for configuration and troubleshooting common issues.
- Key aspects covered include how Kerberos tickets and keytabs are used, the SASL and GSSAPI protocols that enable authenticated RPC, and approaches like delegation tokens and proxy users that handle special cases like long-running jobs.
JRuby allows developers to write plugins for data processing systems like Norikra and Embulk in Ruby while taking advantage of Java libraries and the JVM. Norikra is a stream processing system that allows SQL queries over data streams. It is written in JRuby and uses the Java Esper library. Embulk is an open-source ETL tool that loads data between databases and file formats using plugins. Both systems use a plugin architecture where plugins can be written in JRuby or Java and are distributed as Ruby gems. This allows for a pluggable ecosystem that benefits from Ruby's productivity while utilizing Java libraries and the JVM's performance.
Logging for Production Systems in The Container Era discusses how to effectively collect and analyze logs and metrics in microservices-based container environments. It introduces Fluentd as a centralized log collection service that supports pluggable input/output, buffering, and aggregation. Fluentd allows collecting logs from containers and routing them to storage systems like Kafka, HDFS and Elasticsearch. It also supports parsing, filtering and enriching log data through plugins.
Memcached is a free and open-source distributed memory caching system that can be used to speed up dynamic web applications by reducing database load. It stores objects in memory to return frequently or recently used results very quickly. Common things to cache include query results, objects with heavy calculations, and anything that takes time to generate like database calls, API calls, or page rendering. The memcached client knows all memcached servers and hashes keys to determine which server to store or retrieve each object from. Objects are stored using keys and have a maximum size of 1MB. Commands like get, set, add, delete are used to interact with the cache.
This document provides instructions for setting up Apache Kafka and Spark Streaming to process streaming data from Kafka with Spark. It describes how to install Zookeeper and Kafka, create a Kafka topic, produce and consume messages, and run the KafkaWordCount Spark Streaming example application to perform word count on the streaming data from Kafka. It also explains the different processing semantics supported by Spark Streaming for Kafka integration.
Fast as C: How to Write Really Terrible JavaCharles Nutter
For years we’ve been told that the JVM’s amazing optimizers can take your running code and make it “fast” or “as fast as C++” or “as fast as C”…or sometimes “faster than C”. And yet we don’t often see this happen in practice, due in large part to (good and bad) development patterns that have taken hold in the Java world.
In this talk, we’ll explore the main reasons why Java code rarely runs as fast as C or C++ and how you can write really bad Java code that the JVM will do a better job of optimizing. We’ll take some popular microbenchmarks and burn them to the ground, monitoring JIT logs and assembly dumps along the way.
PostgreSQL as seen by Rubyists (Kaigi on Rails 2022)Андрей Новиков
PostgreSQL has become the most popular RDBMS in the Ruby ecosystem in the last decade. It has a great set of built-in features, including a variety of versatile data types, both common and very specific.
But when we load data from the database to our application code, we're working with Ruby data types: classes from the standard library, Rails, or other gems. So while they can seem to be the same as their PostgreSQL counterparts, they are not absolutely identical, and sometimes that could lead to surprising behavior.
In this talk, I would like to explore the power of data types in PostgreSQL and Ruby and how to work with them properly to use both Ruby and PostgreSQL on 100% of their power!
C has become pretty old school, but the way C developers now and throughout its history have used C, extended C is the foundation of a amazing dynamic languages. This talk describes some dynamic C data structures and also gives an overview of developing a language on top of these tools.
The document discusses implementing cryptography security on a Cortex-M3 microcontroller. It outlines motivations for embedded security and requirements from standards like NIST FIPS-140-2. Details are provided on implementing cryptographic primitives like SHA-256 and HMAC using the ARM architecture. The design of a lightweight embedded security library called GV_Securelib is also described.
Presentation with a brief history of C, C++ and their ancestors along with an introduction to latest version C++11 and futures such as C++17. The presentation covers applications that use C++, C++11 compilers such as LLVM/Clang, some of the new language features in C++11 and C++17 and examples of modern idioms such as the new form compressions, initializer lists, lambdas, compile time type identification, improved memory management and improved standard library (threads, math, random, chrono, etc). (less == more) || (more == more)
Data Wars: The Bloody Enterprise strikes backVictor_Cr
I would like to describe such cases when we create problems for "future us" just by an accident. I will show how different Java data types can ease or increase the pain in supporting the application later. Most common pitfals and tricky corner cases you probably have never thought about.
Introduction to source{d} Engine and source{d} Lookout source{d}
Join us for a presentation and demo of source{d} Engine and source{d} Lookout. Combining code retrieval, language agnostic parsing, and git management tools with familiar APIs parsing, source{d} Engine simplifies code analysis. source{d} Lookout, a service for assisted code review that enables running custom code analyzers on GitHub pull requests.
This document discusses a technique called PK chunking that can be used to efficiently query and retrieve data from very large Salesforce objects with millions of records. PK chunking works by dividing the object into parallel "chunks" using the primary key (PK) IDs, then querying each chunk separately to avoid timeouts and selective query errors. The document explains two implementations of PK chunking - Query Locator PK Chunking (QLPK) which uses the Salesforce query locator to define ID ranges, and Base62PK Chunking which extrapolates ID ranges from the first and last IDs. It provides examples of how to implement PK chunking using JavaScript remoting actions to query the chunks in parallel on the client.
This document discusses moving from C to Go and compares various aspects of memory allocation between the two languages. It covers how arrays are values in Go rather than pointers, stack allocation versus heap allocation, and Go's garbage collector. It also provides an overview of the Go toolchain including supported architectures and compares performance of GCCGO versus the standard Go compiler. Finally, it explains Go's M:N user-space scheduler model and how goroutines are scheduled across logical processors.
Wolltest du schon immer die Vorteile und Ideen von Scala in deinen Java oder Kotlin Projekten nutzen? Dann ist Vavr (ehemals Javaslang) genau die richtige Bibliothek für dich.
Anhand echter Projektbeispiele schauen wir uns den Nutzen an, den Vavr mit seinen syntaktischen Erweiterungen und Features bei der täglichen Arbeit bietet. Wir schauen uns Value Types, echte funktionale Datentypen an und werden lernen, wie wir Exceptions sinnvoller behandeln können. Alles für besser wartbaren und sauberen Code!
Vavr bietet die Möglichkeit, die Vorteile objekt-funktionaler Programmierung zu nutzen, ohne Java den Rücken kehren zu müssen.
HBase Data Modeling and Access Patterns with Kite SDKHBaseCon
This document discusses the Kite SDK and how it provides a higher-level API for developing Hadoop data applications. It introduces the Kite Datasets module, which defines a unified storage interface for datasets. It describes how Kite implements partitioning strategies to map data entities to storage partitions, and column mappings to define how data fields are stored in HBase tables. The document provides examples of using Kite datasets to randomly access and update data stored in HBase.
This document discusses hash functions and their applications. It covers hash function properties, popular hash functions used in applications like hash tables and sets, and how to evaluate hash functions. It also discusses Bloom filters, including how to tune them, and HashFile, a hash-oriented storage structure that provides constant-time lookups from disk. The document concludes with future work ideas like implementing new hash functions and extending HashFile capabilities.
The document outlines an agenda for a presentation on Apache Cassandra. It includes sections on modeling, hardware requirements, configuration tuning, security, adding and removing nodes, multi-datacenter setups, compaction, data purging, upgrades, repairs, backups, monitoring, tools, and common problems. The presenters are Jérôme Mainaud and Alexander Dejanovski, who will go over fundamentals, best practices, and war stories about using Cassandra in production environments.
Java collections such as ArrayList and HashMap were discussed. Common problems with collections like concurrent modification exceptions were presented along with scenarios demonstrating check-then-act bugs. Immutable and persistent collections were described as alternatives that reduce bugs by restricting state changes. Tradeoffs between different collection implementations like memory usage and performance were examined.
Jose Selvi - Side-Channels Uncovered [rootedvlc2018]RootedCON
En los últimos años, el término "side-channel" a pasado de ser un concepto únicamente conocido en el sector de hardware hacking a ser un término popular dentro de la industria debido a las vulnerabilidades que se han ido publicando. CRIME, BREACH o FIESTA son claros ejemplos de vulnerabilidades que explotan un side-channel en TLS. Más recientemente, también hemos visto vulnerabilidades empleando este mismo concepto en procesadores, como Spectre o Meltdown.
En esta charla, repasaremos el concepto de "side-channel" y haremos un repaso por las diferentes vulnerabilidades que se han ido publicando a lo largo de estos últimos años, explicando en que consisten y que limitaciones tienen.
C# 7.x What's new and what's coming with C# 8Christian Nagel
C# is extended in a fast pace – with new features allow to reduce the code you need to write, offer more safety, and gives better performance – and you still write safe code. In this session you are introduced to the new C# 7.0-7.3 features including tuples and pattern matching, and learn about the features planned with C# 8.0 such as nullable reference types, extensions for pattern matching, and the new switch expression.
The amount of data collected by applications nowadays is growing at a scary pace. Many of them need to handle billions of users generating and consuming data at an incredible speed. Maybe you are wondering how to create an application like this? What is required? What works best for your project?
In this session we’ll compare popular Java and JVM persistence frameworks for NoSQL databases: Spring Data, Micronaut, Hibernate OGM, Jakarta NoSQL, and GORM. How do they compare, what are the strengths, weaknesses, differences, and similarities? We’ll show each of them with a selection of different NoSQL database systems (Key-Value, Document, Column, Graph).
The data load on applications has increased exponentially in recent years. We know the JVM (Java Virtual Machine) can cope with heavy loads very well yet we often come across the big dilemma: there are tons of persistence frameworks out there but which one performs best for my case? It would normally take ages to evaluate and choose the best fit for your use case. We’ve done those comparisons for you.
String Comparison Surprises: Did Postgres lose my data?Jeremy Schneider
Comparisons are fundamental to computing - and comparing strings is not nearly as straightforward as you might think. Come learn about the history, nuance and surprises of “putting words in order” that you never knew existed in computer science, and how that nuance impacts both general programming and SQL programming. Next, walk through a few actual scenarios and demonstrations using PostgreSQL as a user and administrator, which you can re-run yourself later for further study, including one way you could easily corrupt your self-managed PostgreSQL database if you aren't prepared. Finally we’ll dive into an explanation of the surprising behaviors we saw in PostgreSQL, and learn more about user and administrative features PostgreSQL provides related to localized string comparison.
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...kalichargn70th171
In today's business landscape, digital integration is ubiquitous, demanding swift innovation as a necessity rather than a luxury. In a fiercely competitive market with heightened customer expectations, the timely launch of flawless digital products is crucial for both acquisition and retention—any delay risks ceding market share to competitors.
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
Consistent toolbox talks are critical for maintaining workplace safety, as they provide regular opportunities to address specific hazards and reinforce safe practices.
These brief, focused sessions ensure that safety is a continual conversation rather than a one-time event, which helps keep safety protocols fresh in employees' minds. Studies have shown that shorter, more frequent training sessions are more effective for retention and behavior change compared to longer, infrequent sessions.
Engaging workers regularly, toolbox talks promote a culture of safety, empower employees to voice concerns, and ultimately reduce the likelihood of accidents and injuries on site.
The traditional method of conducting safety talks with paper documents and lengthy meetings is not only time-consuming but also less effective. Manual tracking of attendance and compliance is prone to errors and inconsistencies, leading to gaps in safety communication and potential non-compliance with OSHA regulations. Switching to a digital solution like Safelyio offers significant advantages.
Safelyio automates the delivery and documentation of safety talks, ensuring consistency and accessibility. The microlearning approach breaks down complex safety protocols into manageable, bite-sized pieces, making it easier for employees to absorb and retain information.
This method minimizes disruptions to work schedules, eliminates the hassle of paperwork, and ensures that all safety communications are tracked and recorded accurately. Ultimately, using a digital platform like Safelyio enhances engagement, compliance, and overall safety performance on site. https://safelyio.com/
UI5con 2024 - Bring Your Own Design SystemPeter Muessig
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
Liberarsi dai framework con i Web Component.pptxMassimo Artizzu
In Italian
Presentazione sulle feature e l'utilizzo dei Web Component nell sviluppo di pagine e applicazioni web. Racconto delle ragioni storiche dell'avvento dei Web Component. Evidenziazione dei vantaggi e delle sfide poste, indicazione delle best practices, con particolare accento sulla possibilità di usare web component per facilitare la migrazione delle proprie applicazioni verso nuovi stack tecnologici.
Using Query Store in Azure PostgreSQL to Understand Query PerformanceGrant Fritchey
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Malibou Pitch Deck For Its €3M Seed Roundsjcobrien
French start-up Malibou raised a €3 million Seed Round to develop its payroll and human resources
management platform for VSEs and SMEs. The financing round was led by investors Breega, Y Combinator, and FCVC.
2. @tall_chris#Devoxx #TCoffheap
Who Am I?
• Trained as a Physicist, clearly not trained as a Computer
Scientist.
• 4Years Doing Unnatural Things With Bytecode In Academia
• 3Years Doing Unnatural Things With Bytecode For Money
• 4Years Doing Unnatural Things With ByteBuffers
• 11Years Doing Java Development
• Software Engineer working at Terracotta (Software AG)
4. @tall_chris#Devoxx #TCoffheap
A Bit of History
2010 Started development as a caching ‘tier’ within Ehcache.
2011 Integrated as a caching tier in front of Oracle BDB in the
Terracotta Server.
2013 Legal complications push it in to service as the primary
storage for the Terracotta Server.
2015 Open Sourced (https://github.com/Terracotta-OSS/
offheap-store).
7. @tall_chris#Devoxx #TCoffheap
Problem Statement
• “a lot of caching” leads to
• a lot of heap, which leads to,
• a lot of work for the garbage collector, which leads to,
• a lot of GC pausing/overhead”
• The situation is markedly better now than when the bulk of this
library was written. (Please don’t tell my employer I said that)
8. @tall_chris#Devoxx #TCoffheap
Map/Cache Best Practices
• Immutable Keys
• please do this!
• ImmutableValues
• please do this!
• So with immutability everywhere, who cares about object
identity?
• If I don’t need object identity, do I need a heap?
• If I don’t need a heap, do I need a garbage collector?
9. @tall_chris#Devoxx #TCoffheap
Solution
• Replace heavy (large) map/cache usage with an ‘outside the
heap’ but ‘inside the process’ implementation.
• Benefits at two scales:
• At moderate scale, the GC offload reduces overheads.
• At large scale, we can still function: -Xmx6T
• Caveats
• Marshalling/unmarshalling costs time (and CPU)
• Trading away average latency to control the tail.
44. @tall_chris#Devoxx #TCoffheap
Options with 64 bits available
• 64 bit combined pointer
• 32 bit key pointer & 32 bit value pointer
• int key directly + 32 bit pointer
• long key directly + 32 bit pointer
• …anything else you like
48. @tall_chris#Devoxx #TCoffheap
A Native Heap Allocator
• malloc/free performed using a Java port of dlmalloc
• http://g.oswego.edu/dl/html/malloc.html
• Works well for our use cases as we do not generally control
or even know the malloc size distribution.
50. @tall_chris#Devoxx #TCoffheap
“Java Serialization Sucks”
• Serialization is self describing.
• It supports
• object identity
• cycles
• complex versioning
• Pretty heavyweight, especially for short streams…
• …but it’s the default serialization mechanism available in
Ehcache 2.x
51. @tall_chris#Devoxx #TCoffheap
“Java Serialization Sucks”
• serialize(new Integer(42))
• results in these 81 bytes:
0 1 2 3 4 5 6 7 8 9 A B C D E F
0 AC ED 00 05 73 72 00 11 6A 61 76 61 2E 6C 61 6E
1 67 2E 49 6E 74 65 67 65 72 12 E2 A0 A4 F7 81 87
2 38 02 00 01 49 00 05 76 61 6C 75 65 78 72 00 10
3 6A 61 76 61 2E 6C 61 6E 67 2E 4E 75 6D 62 65 72
4 86 AC 95 1D 0B 94 E0 8B 02 00 00 78 70 00 00 00
5 2A
52. @tall_chris#Devoxx #TCoffheap
OffHeap’s Serialization Sucks Less?
• serialize(new Integer(42))
• results in 22 bytes
0 1 2 3 4 5 6 7 8 9 A B C D E F
0 AC ED 00 05 73 72 00 00 00 00 78 72 00 00 00 01
1 78 70 00 00 00 2A
2
3
4
5
54. @tall_chris#Devoxx #TCoffheap
With some structure
STREAM_MAGIC STREAM_VERSION
TC_OBJECT
TC_CLASSDESC descriptor(0)
TC_END_BLOCKDATA
TC_CLASSDESC descriptor(1)
TC_END_BLOCKDATA
TC_NULL
0000002A
55. @tall_chris#Devoxx #TCoffheap
Where did the 59 bytes go?
• How many types are in my map?
• All keys the same type: really common
• All values the same type: fairly common
• Stick those common ObjectStreamClass instances in a look
aside structure
• Map<Integer, ObjectStreamClass> for reading streams
• Map<SerializableDataKey, Integer> for writing streams
59. @tall_chris#Devoxx #TCoffheap
j.u.c.ConcurrentMap
• What does a concurrent map provide?
• happens-before relationship: “actions in a thread prior to placing an
object into a ConcurrentMap as a key or value happen-before actions
subsequent to the access or removal of that object from the
ConcurrentMap in another thread”
• atomic operations: “…except that the action is performed atomically.”
• What do we want?
• concurrent access (readers and writers)
60. @tall_chris#Devoxx #TCoffheap
Happens Before Relationships
• volatile write/read
• but not on offheap memory locations
• synchronized
• needs a heap object
• other j.u.c classes (Lock,Atomic…)
• needs a heap object
• There is no way within the JDK to enforce a happens before
relationship between writes/reads of an offheap location…
61. @tall_chris#Devoxx #TCoffheap
No Unsafe please, we’re a library
• Our testing has never shown our offheap implementation to
be a bottleneck in our usages.
• Unnecessary complexity costs $$$
• support
• maintenance
• bugs
65. @tall_chris#Devoxx #TCoffheap
A ‘Concurrent’ Map
✅ happens-before relationship: “actions in a thread prior to
placing an object into a ConcurrentMap as a key or value
happen-before actions subsequent to the access or removal of
that object from the ConcurrentMap in another thread”
✅ atomic operations: “…except that the action is performed
atomically.”
⚠ concurrent access (readers and writers)