Oracle Code One 2018, Birds of a Feather (BOF) Session
Bytecode Manipulation with a Java Agent and Byte Buddy
[BOF5314]
Tuesday, Oct 23, 07:30 PM - 08:15 PM | Moscone West - Room 2005
Have you ever manipulated Java bytecode? There are several bytecode engineering libraries, and Byte Buddy is one of the easiest, and you can also use Java agents, which are related to the Instrumentation class in the java.lang.instrument API. Instrumentation is the addition of bytecode to methods. Because the changes are purely additive, a Java agent does not modify application state or behavior. With Byte Buddy and a Java agent, we can add behaviors to existing classes. This session explains what Java agents and the instrumentation API are, introduces Byte Buddy, and presents sample code that uses a Java agent and Byte Buddy to modify behavior. The presentation will be useful for those who want to start manipulating Java bytecode with Byte Buddy.
11. Byte Buddy
• Usage in major libraries
–Hibernate
–Mockito
–Jackson
–etc.
#oc1bcm11
12. Class<?> subClass = new ByteBuddy()
.subclass(Object.class)
.method(ElementMatchers.named("toString"))
.intercept(FixedValue.value("Hello World!"))
.make()
.load(getClass().getClassLoader())
.getLoaded();
Hello World in Byte Buddy
#oc1bcm12
13. Class<?> subClass = new ByteBuddy()
.subclass(Object.class)
.method(ElementMatchers.named("toString"))
1. Create a subclass of Object
2. Choose the toString()
Hello World in Byte Buddy
#oc1bcm13
20. How to use BM?
1. Directly in your application
2. At build time via Maven/Gradle
3. With Java Agent
– No need for changing code
– Detachable
#oc1bcm20
21. Java Agent?
• Do espionage activities in
your application
– Profiling
– Logging
– Changing the target
application itself
#oc1bcm21
22. Java Agent
• Instrument programs running on the
JVM
–JSR 163: JavaTM Platform Profiling
Architecture
–JVM calls Java Agent method with
proper timing
#oc1bcm22
23. Create a Java Agent
• Just define methods
–premain() Method
–agentmain() Method
#oc1bcm23
24. Create a Java Agent
• premain() Method
–Gets called before main() method
• agentmain() Method
–Gets called when the agent is attached
after JVM startup
#oc1bcm24
25. With Java Agent
We can run bytecode
manipulation code
at runtime
#oc1bcm25
27. premain()
public static void premain(String agentArgs,
Instrumentation inst)
• Provides opportunity to modify
classes before loading
–Instrumentation class has useful
methods
#oc1bcm27
30. Another Example
• Add outputting logs before executing
a particular method on a particular
class
#oc1bcm30
31. Example
1. Manipulate bytecode before loading
–Use premain()
2. Replace the class file
–Use ByteBuddy
–Not create a subclass
#oc1bcm31
32. public class Target {
public void foo() {
System.out.println("foo!");
}
public static void main(String[] args) {
new Target().foo();
}
}
Target Class
#oc1bcm32
33. TypePool pool = TypePool.Default.ofClassPath();
return new ByteBuddy()
.rebase(pool.describe("c.Target").resolve(),
ClassFileLocator.ForClassLoader
.ofClassPath()
)
Replace the class
#oc1bcm33
34. return new ByteBuddy()
.rebase(...)
• Replace the original implementation with
new one
• Save the original implementation under a
different method name
Replace the class
#oc1bcm34
35. Enhance Existing Method
• ByteBuddy#subclass()
–Override the method
• ByteBuddy#redefine()
–Replace the implementation
• ByteBuddy#rebase()
–Copy&Rename the method
–Replace the implementation
#oc1bcm35
36. TypePool pool = TypePool.Default.ofClassPath();
return new ByteBuddy()
.rebase(pool.describe("c.Target")
.resolve(),
• Can't use a class literal
– JVM loads class before changing the class
• Use the TypePool class and describe()
Replace the class
#oc1bcm36
41. public static class LoggingInterceptor {
public static void intercept(
@Origin Method m) {
...println("Call " + m.getName()...);
}
}
Interceptor Class
#oc1bcm41
46. public byte[] transform(ClassLoader loader,
String className,
Class<?> classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classfileBuffer)
• className – Class name to be loaded
• classfileBuffer – Class file content
transform()
#oc1bcm46
47. public byte[] transform(ClassLoader loader,
String className,
Class<?> classBeingRedefined,
ProtectionDomain protectionDomain,
byte[] classfileBuffer)
• Return the class file content
transform()
#oc1bcm47
48. Demo 3
• Add outputting logs before executing
a method
#oc1bcm48
50. Features in ByteBuddy
• Add fields/methods
• Manipulate stack
• Generate CONDY bytecode
– Constant Dynamic – JEP 309
• Simplify Java Agent with AgentBuilder
• lots of other features
#oc1bcm50
55. Without -javaagent
• -javaagent option can be removed
–Call ByteBuddyAgent.install()
–Call AgentBuilder#installOnByteBuddyAgent()
• instead of installOn()
–Need to add byte-buddy-agent.jar
#oc1bcm55
58. Wrap Up
• BM is fun and close to us!
–Many libraries are using
• Application Examples
–Hot Swap
–Advanced Dynamic Proxy
#oc1bcm58
Notes de l'éditeur
Hello, everyone! Today, I'd like to talk about manipulating Java bytecode. I'll introduce Java Agent and a library called Byte Buddy.
Let me briefly introduce myself. I'm Koichi Sakata. I'm a KanJava JUG leader. Kan means Kansai region in Japan. It includes Kyoto. Do you know Kyoto? Kyoto is visited by many tourists. I became a Java champion this year. I work at PONOS Corporation, that is a mobile game company.
This session is for beginners, not for experts.
Moving on to our main topic. Java bytecode is generated by the javac compiler. It's in a class file. It's an intermediate representation of Java programs. We can visualize bytecode with javap command.
What is bytecode manipulation? It's to edit bytecode, such as add, delete and replace bytecode.
Why do we manipulate bytecode? Because it's fun!
But not only fun. It's also useful to change code at runtime. For example, Hibernate uses bytecode manipulation to generate entity objects, and Mockito uses it to create mock objects. Of course at runtime.
It's hard to change bytecode by hand. Because a class file is a binary file and JVM has a verification process for a class file.
So there are many libraries for bytecode manipulation. Byte Buddy, Byteman, Javassist, cglib, ASM and BCEL. Today I'll introduce Byte Buddy.
Why Byte Buddy? Byte Buddy has a Easy-to-use API, supports Java 11 already and won a Duke's Choice Award.
Many major libraries are using Byte Buddy, such as Hibernate, Mockito, Jackson, etc.
This is a hello world program with Byte Buddy. It creates a new class that extends Object class, and overrides toString() method to return "Hello World". I'll explain the details of this code.
First, we create a subclass of the Object class and choose the toString() method.
Next, we intercept the metod to return a fixed string value "Hello World".
Make method builds a new type. The new type isn't loaded by a classloader yet, so the type is an unloaded class. Then the classloader loads the type. Finally we can get the new loaded class object. It's easy to manipulate bytecode with Byte Buddy. That's the reason why Byte Buddy is so popular.
OK, I explained all parts of this code.
Let's run the hello world.
This code is a bit different from the previous code. I'll save a generated class to a file and outputs the method return.
(run HelloWorld)
Let's see the generated file. It is in a temp directory. Byte Buddy names the dynamic class randomly.
By the way, the demo code used today is available on GitHub. Just access "bitly/oc1bcm"
Now I'd like to show code in Mockito. Mockito is a famous mocking framework.
This method creates a mock class. As you can see, it's similar to our hello world example.
How to use? You can run bytecode manipulation code directly in your application. But you need to change your code. When you choose the second option, you need to rebuild. So I'd like to introduce Java Agent. It doesn't require the change. It's also detachable.
What is Java Agent? The word "Agent" invokes the "Mission: Impossible" spy series. Java Agent do espionage activities in your application, like profiling, logging and even changing the behavior of the application itself.
Java Agent can instrument programs running on the JVM. It's defined as JSR 163. The JVM calls Java Agent methods with proper timing.
To create a Java Agent, just define one of the these methods or both, premain() and agentmain().
Premain() method is called by the JVM before the application's main method(). Agentmain() is called when the agent is attached after the JVM has started.
So with Java Agent, we can run bytecode manipulation code at runtime.
To create a Java Agent, first, define the Agent Class method. Second, write a MANIFEST.MF file. Third, package the Agent class to a JAR file. Finally, run java command with javaagent option.
A signature of premain() is "public static void premain()". We can have an opportunity to modify classes before the class is loaded. Instrumentation class in method parameters provides useful methods to modify classes.
In the manifest file, we specify the Agent class and set the path to the Byte Buddy's JAR file.
Let's create a simple Java Agent.
The Agent class is a Java agent class. It has the premain() method.
This is the Main class.
Run the main() method with javaagent option. As you can see, premain() executed before main method.
Let's combine Java Agent with Byte Buddy. I'll show another example. It adds outputting logs before executing a particular method on a particular class.
I'll do two things. First, define the premain() method in a class. Second, write code in premain() method to replace the class file to output a log. Note that I don't create a subclass.
This is the Target class. We want to output a log when foo() method is called.
Next, replace the class with Byte Buddy. Let me explain the details of this code.
This code uses rebase() method, instead of previous subclass() method. Rebase() method doesn't create a subclass. It replaces the original implementation with new one and saves it under a different method name.
Byte Buddy provides three method to enhance existing methods. Subclass() creates a subclass and overrides the method. Redefine() replaces the implementation with new one. Rebase() also replaces it and saves the original implementation.
Note that we can't use a class literal. If we use a class literal for the Target class, the JVM will load the class before changing the class. We can use the TypePool class and its describe() method, to represent the class without the class literal.
Also, note that when using unloaded classes, we need to specify a ClassFileLocator. It allows to locate a class file. In this example, we simply create a class file locator with ofClassPath() method. It scans the running application's class path for class files.
This time I'd like to delegate outputting to another class, LoggingInterceptor class.
To do so, We can use MethodDelegation.to() method. We simply specify a class to delegate. A best match method is chosen automatically by Byte Buddy.
After the method is invoked, the original call will continue, thanks to andThen() method with SuperMethodCall.INSTANCE. SuperMethodCall.INSTANCE represents the original method.
This is the interceptor class. This code outputs the intecepted method name.
In delegated method, various annotations can be used. The Origin annotation is a reference to the original method or constructor. So we can get the method object of the original method.
Other annotations are shown here.
More details can be found in the Byte Buddy's website.
Go back to the premain(). How do we run that code in Java Agent? Instrument API provides ClassFileTransformer class to change bytecode. To manipulate bytecode in Java Agent, We need to override transform(). Then pass the ClassFileTransformer instance to addTransformer() of the Instrumentation class. The transform() will be called by the JVM before loading.
Let's go on to the transform() method. We can call the manipulation code in transform() method.
In parameters, class name is the name to be loaded. Class file buffer is the content of the class file.
We return the class file content from the transform().
Finally we can run the demo. Let's see the complete code.
(Show LoggingAgent class)
(run Target class)
OK, foo() is intercepted.
So we covered the basics of Byte Buddy and Java Agent.
Of course, there are many features in Byte Buddy. Add fields and methods, manipulate even stacks. Also, we can generate CONDY code with Byte Buddy. Have you ever heard about condy? CONDY means constant dynamic. It defines as JEP 309. It's included from Java 11, but the Java compiler doesn't use CONDY yet. We can try to use CONDY directly with Byte Buddy.
Acutually Byte Buddy provides AgentBuilder class to simplify Java Agent. We can rewrite the previous Logging Agent with AgentBuilder.
Just use AgentBuilder class instead of ByteBuddy class, and use AgentBuilder.Transformer instead of ClassFileTransformer. Call install() method at the end.
Use AgentBuilder, Use AgentBuilder.Transformer and call installOn().
Then We can use builder object to intercept the method. Let's see the complete code.
(show LoggingAgent class)
This code is more readable, isn't it?
(mvn package & run Target class)
Additionally, when we use AgentBuilder, we can remove javaagent option. Just call ByteBuddyAgent's install() and AgentBuilder's installOnByteBuddyAgent() instead of installOn() method. To use ByteBuddyAgent, we need to add byte-buddy-agent JAR file to the class path.
Call ByteBuddyAgent.install() and AgentBuidler's installOnByteBuddyAgent().
(show LoggingMainWithoutAgent class)
Run this code without javaagent option.
(run LoggingMainWithoutAgent)
Let's wrap up. Bytecode manipulation is fun, and it's close to us. I'd like to stress this point. Many famous libraries are using bytecode manipulation. As application examples, there are How Swap, which swap classes at runtime, and advanced dynamic proxy.