4. diagnosis
jmap –histo pid
num #instances #bytes class name
----------------------------------------------
1: 1001744 246906600 [C //char array
2: 2855952 137085696 edu.emory.mathcs.backport.java.util.concurrent.ConcurrentHashMap$Segment
3: 282338 115692192 [I // int array
4: 677362 102508648 [Ljava.util.HashMap$Entry;
5: 192117 99164392 [B // byte array
6: 2856122 91732120 [Ledu.emory.mathcs.backport.java.util.concurrent.ConcurrentHashMap$HashEntry;
7: 2855952 91390464 edu.emory.mathcs.backport.java.util.concurrent.locks.ReentrantLock$NonfairSync
8: 1337498 53499920 java.lang.String
9: 433634 43130120 [Ljava.lang.Object;
10: 597908 38266112 java.util.HashMap
From the histogram, the memory leak is caused by Axis2.
1) The main change in this release is using web service client (axis2 1.4.1)
2) char[] / edu.emory.mathcs.backport.java.util.concurrent.ConcurrentHashMap
3) Google ‘axis2 1.4.1 concurrentHashMap memory leak‘
5. diagnosis
jmap –dump:file=cdump pid
• We got a 2.39G memory heap dump
– Very fast, less than 30 seconds (because 8G
memory in production)
• Problem : too big to open in dev box
– Need enough physical memory
– (May) need 64 bit OS
– Try Jhat/Jprofile/YJP/Jmat eclipse plugin
(windows/2G)
Open by Jmat standalone application
19. Axis2 1.4.1 memory leak
• Cause: Client Finalize (no change)
//AsixConiguration
public AxisServiceGroup removeServiceGroup(String serviceGroupName) throws AxisFault {
…
Iterator services = axisServiceGroup.getServices();
while (services.hasNext()) {
AxisService axisService = (AxisService) services.next();
allServices.remove(axisService.getName());
…
for (Iterator iter = axisService.getEndpoints().keySet().iterator(); iter.
key = serviceName + "." + (String)iter.next();
this.allEndpoints.remove(key);
}
…
}
Servce are already removed from ServiceGroup in Stub cleanup
…
}
The while loop would never enter
20. Axis2 1.4.1 memory leak
• Cause
– Stub cleanup and ServiceClient cleanup have
dependency
– Can not call in below order
Stub.cleanup
ServiceClient.cleanup
• Two are two memory leak bugs in 1.4, 1.4.1 only
fix 1 bug
22. Implement issue
• Forget to cleanup
– Container object: add only, no remove
– Resource object: apply only, no return/close
• Cleanup dependency
– One object cleanup depend on other objects
– Two object cleanup/two method has order
dependency
23. Design Issue
• AxisConfiguration is a global shared object
– Usually only 1 instance even in client side.
• Purpose for put service/endpoint map in this
global object AxisConfiguration ?
24. Design Issue
• Message Dispatch in server side
// RequestURIBasedServiceDispatcher
public AxisService findService(MessageContext messageContext) throws AxisFault {
AxisConfiguration registry = configurationContext.getAxisConfiguration();
AxisService axisService = registry.getService(values[0]);
// If the axisService is not null we get the binding that the request came to add
// add it as a property to the messageContext
if (axisService != null) {
Map endpoints = axisService.getEndpoints();
if (endpoints != null) {
if (endpoints.size() == 1) {
messageContext.setProperty(WSDL2Constants.ENDPOINT_LOCAL_NAME,
endpoints.get(axisService.getEndpointName()));
} else {
String endpointName = values[0].substring(values[0].indexOf(".") + 1);
messageContext.setProperty(WSDL2Constants.ENDPOINT_LOCAL_NAME,
endpoints.get(endpointName));
}
}
}
return axisService;
}
25. Design Issue
• Service is ‘singleton’ in server side
// AxisConfiguration
public AxisService getService(String name) throws AxisFault {
AxisService axisService = (AxisService) allServices.get(name);
if (axisService != null) {
if (axisService.isActive()) {
return axisService;
} else {
throw new AxisFault(Messages
.getMessage("serviceinactive", name));
}
} else {
axisService = (AxisService) allEndpoints.get(name);
if (axisService != null) {
if (axisService.isActive()) {
return axisService;
} else {
throw new AxisFault(Messages
.getMessage("serviceinactive", name));
}
}
}
return null;
}
26. Design Issue
• Client side
– (Usually) New instance every method invocation
– No need for message routing
• Why still register in Axi2Configuration ?
axisConfig = configContext.getAxisConfiguration();
if (axisService == null) {
axisService = createAnonymousService();
}
this.axisService = axisService;
if (axisConfig.getService(axisService.getName()) == null) {
axisService.setClientSide(true);
axisConfig.addService(axisService);
} else {
…
}
…