Solving weblogic.utils.io.Chunk objects.Out of Memory Exception | Weblogic

Instance has many durable subscribers to a distributed topic. It has a distributed topic with 3 members on ms1,ms2 and ms3, ms1 has 100 durable subscribers .

Message size is between 6 to 10MB.

Issue:

When a message is delivered to the topic an out of memory errors occurs. Heap Profiling shows that 98% of the heap is used by byte[] of 4096 bytes (4096=chunk size) referenced by weblogic.utils.io.Chunk objects.Out of Memory Exception

Logs:

<Oct 2, 2011 10:08:00 AM BST> <Info> <Health> <BEA-310002> <13% of the total memory in the server is free>
<Oct 2, 2011 10:25:18 AM BST> <Error> <Kernel> <BEA-000802> <ExecuteRequest failed
java.lang.OutOfMemoryError: Java heap space.
java.lang.OutOfMemoryError: Java heap space
at weblogic.utils.io.Chunk.<init>(Chunk.java:293)
at weblogic.utils.io.Chunk.getChunk(Chunk.java:141)
at
weblogic.utils.io.ChunkedOutputStream.advance(ChunkedOutputStream.java:52)
at weblogic.utils.io.ChunkedOutputStream.write(ChunkedOutputStream.java:42)
at
weblogic.utils.io.ChunkedDataOutputStream.writeUTF8(ChunkedDataOutputStream.
java:298)
at
weblogic.utils.io.ChunkedDataOutputStream.writeUTF8Chars(ChunkedDataOutputSt
ream.java:285)
at
weblogic.utils.io.ChunkedDataOutputStream.writeUTF8String(ChunkedDataOutputS
tream.java:236)
at
weblogic.utils.io.ChunkedDataOutputStream.writeUTF8(ChunkedDataOutputStream.
java:208)
at
weblogic.jms.common.TextMessageImpl.writeExternal(TextMessageImpl.java:294)
at weblogic.jms.common.JMSPushRequest.writeExternal(JMSPushRequest.java:223)
at weblogic.messaging.dispatcher.Request.writeShortened(Request.java:1321)
at
weblogic.messaging.dispatcher.DispatcherObjectHandler.writeRequest(Dispatche
rObjectHandler.java:31)
at
weblogic.messaging.dispatcher.DispatcherProxy.marshal(DispatcherProxy.java:2
14)
at
weblogic.messaging.dispatcher.DispatcherProxy.marshal(DispatcherProxy.java:1
71)
at
weblogic.messaging.dispatcher.DispatcherProxy.dispatchOneWay(DispatcherProxy
.java:149)
at
weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchNoReply(Dispatc
herWrapperState.java:182)
at
weblogic.jms.dispatcher.DispatcherAdapter.dispatchNoReply(DispatcherAdapter.
java:27)
at
weblogic.jms.common.JMSServerUtilities.anonDispatchNoReply(JMSServerUtilitie
s.java:290)
at weblogic.jms.backend.BESessionImpl.pushMessages(BESessionImpl.java:1563)
at weblogic.messaging.util.DeliveryList.run(DeliveryList.java:263)
at
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkMa
nagerImpl.java:516)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

Solution:

Check “Tuning WebLogic JMS” docs try enabling JMS Paging as mentioned in this doc and check Tuning for Large Messages section

http://download.oracle.com/docs/cd/E21764_01/web.1111/e13814/jmstuning.htm#i1157046

Other useful guides:
* JMS Troubleshooting docs:

http://download.oracle.com/docs/cd/E21764_01/web.1111/e13738/troubleshoot.htm

The “Chunks” that were measured are used during the time the server is pushing messages to a remote receiver or getting a message from a remote producer.
Chunks are fixed size network buffers that are pooled internally at WebLogic’s T3/RJVM transport layer.
Receive side might be the problem in this case since typical pub/sub applications have higher aggregate receive rates than publish rates.
For synchronous receivers in a default configuration, the chunk memory usage should worst case correspond to 100 * 10MB = 1000MB (since there are 100 receivers and max message size is 10MB).

Assuming a default configuration means that each synchronous receiver will have at most one message pushed to it at a time.
For asynchronous receivers in a default configuration, up to 10 messages may be pushed to a receiver at any one time, so the chunk memory usage in the setup will worse case to something like 10 * 10MB * 100 = 10000 MB.

This can be reduced by tuning down the connection factory “MessagesMaximum” setting from 10 to 1, leading to a worse case of 1000MB.
In addition, the overall message size can sometimes be reduced by enabling message compression.

If compression isn’t an option, then increasing JVM heap would help.

In case of any ©Copyright or missing credits issue please check CopyRights page for faster resolutions.