java.io.ioexception too many open files weblogic

If you see below error while starting weblogic managed or admin server then follow below given instructions to fix the issue.

java.io.IOException: Error from open(), Too many open files, errno=24
at
weblogic.store.io.file.direct.DirectIONative.openConsiderLock(Native Method)
at
weblogic.store.io.file.direct.DirectFileChannel.<init>(DirectFileChannel.java:54)
at
weblogic.store.io.file.direct.DirectIOManager.open(DirectIOManager.java:179)
at weblogic.store.io.file.StoreFile.openInternal(StoreFile.java:124)
at weblogic.store.io.file.StoreFile.openDirect(StoreFile.java:168)

*java.io.FileNotFoundException:
/opt/oracle/user_projects/domains/dg_domain/servers/AdminServer/logs/access.log
(Too many open files)*
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
at
com.bea.logging.RotatingFileOutputStream.open(RotatingFileOutputStream.java:263)
at
com.bea.logging.RotatingFileOutputStream.initialize(RotatingFileOutputStream.java:324)
at
com.bea.logging.RotatingFileOutputStream.initialize(RotatingFileOutputStream.java:313)
at
com.bea.logging.RotatingFileOutputStream.<init>(RotatingFileOutputStream.java:116)
at
weblogic.servlet.logging.LogManagerHttp$HttpRotatingFileOutputStream.<init>(LogManagerHttp.java:140)
at
weblogic.servlet.logging.LogManagerHttp.getLogOutputStream(LogManagerHttp.java:44)
at
weblogic.servlet.logging.LogManagerHttp.initLoggers(LogManagerHttp.java:76)
at
weblogic.servlet.logging.LogManagerHttp.start(LogManagerHttp.java:58)
at weblogic.servlet.internal.HttpServer.start(HttpServer.java:290)
at
weblogic.servlet.internal.WebService.startWebServers(WebService.java:138)
at weblogic.servlet.internal.WebService.start(WebService.java:97)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
<Apr 2, 2012 11:22:40 PM EDT> <Critical> <WebLogicServer> <BEA-000386>
<Server subsystem failed. Reason: java.lang.NullPointerException
java.lang.NullPointerException
at
weblogic.servlet.logging.LogManagerHttp.initLoggers(LogManagerHttp.java:79)
at
weblogic.servlet.logging.LogManagerHttp.start(LogManagerHttp.java:58)
at weblogic.servlet.internal.HttpServer.start(HttpServer.java:290)
at
weblogic.servlet.internal.WebService.startWebServers(WebService.java:138)
at weblogic.servlet.internal.WebService.start(WebService.java:97)
Truncated. see log file for complete stacktrace
<Apr 2, 2012 11:22:40 PM EDT> <Notice> <WebLogicServer> <BEA-000365>
<Server state changed to FAILED>
<Apr 2, 2012 11:22:40 PM EDT> <Error> <WebLogicServer> <BEA-000383>
<Apr 2, 2012 11:22:40 PM EDT> <Notice> <WebLogicServer> <BEA-000365>
<Server state changed to FORCE_SHUTTING_DOWN>

If you see ths kind of errors in weblogic domain logs, I suggest to check the following point:

– Check the values with “ulimit -a” with linux user that startup weblogic instances with objective to see that value (“open files (-n)1024”);

– Run “lsof” command to see what kind of files are opened with with linux user that startup weblogic instances (“lsof |grep <linux_user>”)

Some considerations:

If the value for open files is 1024, try to grow up to another value, like :32768.

-if the value looks like 32768, you need to analyze lsof results to see what kind of files are open and follow that situation with Development Team.

– Go to limit.conf file and increase the hard and soft values(Needs a reboot of machine):

* hard nofile 65535
* soft nofile 4096
* hard nproc 16384
* soft nproc 2047

To propagate the changes to all labels below can be done followed by a system reboot to make the changes take effect

– modify the /etc/pam.d/system-auth with the following entry

session required /lib/security/$ISA/pam_limits.so

– modify /etc/pam.d/login, /etc/pam.d/su, /etc/pam.d/sshd, /etc/pam.d/gdm

session required pam_limits.so

A system reboot is required post this change.

- For more help Can refer to oracle support document How To Troubleshoot Too Many Open Files Problems (Doc ID 867492.1)

In case of any ©Copyright or missing credits issue please check CopyRights page for faster resolutions.