Maximo List Archive

This is an archive of the Maximo Yahoo Community. The content of this pages may be a sometimes obsolete so please check post dates.
Thanks to the community owner Christopher Wanko for providing the content.

Wepsphere - MxServer Slow performance

From: Brian Scott (2011-07-27 01:19)

If you have 3 app servers in your was node the box it runs on will need at least 5GB ram.
Set the heap sizes for DMgr to 256 512 and each app server to 512 1560
Are you running it all in one windows server?
Probably best to have the db2 on it's own box with 4GB min ram
Brian Scott
Service Management Connect
Sent from my iPad
On 26 Jul 2011, at 23:29, shah faisal <dba_develper@yahoo.com> wrote:
> Dear All, Good Morning
>
> DB: DB2
> OS: Windows 2003 sp2
> Maximo 7.1
>
> My websphere 'MxServer' is extreemly slow and more than 4 times a day I have to restart the server. Kindly help me dig out the problem with solution.
> I have also other application servers such as MxCron,MxTest.
>
> I do not know how to resolve that issue? What steps need to be taken?
>
> I need your urgent help in detail.
>
> Regards
> Shah
>

From: ssc.susser (2011-07-28 17:18)

We were having a huge problem with MxServer locking up. Setting up a cluster really did wonders to increase the up time, but it turns out our problem was with an interaction node in one of our workflows. It had worked previously but when we applied an update the action in the interaction node "routewf" caused an infinite loop. You might want to look into something like that. We ended up tracking down the problem by looking at the wfinstance table, grouping by user, and count() the number of wfinstance records and seeing the massive task calls for that one user.
The pattern for our freezes was very erratic--never happened at the same time of day and not always on the same cluster member.
Are you noticing massive CPU spikes or are you getting OOM errors or both?
--- In MAXIMO@yahoogroups.com, Brian Scott <bpscotty@...> wrote:
>
> If you have 3 app servers in your was node the box it runs on will need at least 5GB ram.
> Set the heap sizes for DMgr to 256 512 and each app server to 512 1560
>
> Are you running it all in one windows server?
>
> Probably best to have the db2 on it's own box with 4GB min ram
>
> Brian Scott
> Service Management Connect
>
> Sent from my iPad
>
> On 26 Jul 2011, at 23:29, shah faisal <dba_develper@...> wrote:
>
> > Dear All, Good Morning
> >
> > DB: DB2
> > OS: Windows 2003 sp2
> > Maximo 7.1
> >
> > My websphere 'MxServer' is extreemly slow and more than 4 times a day I have to restart the server. Kindly help me dig out the problem with solution.
> > I have also other application servers such as MxCron,MxTest.
> >
> > I do not know how to resolve that issue? What steps need to be taken?
> >
> > I need your urgent help in detail.
> >
> > Regards
> > Shah
> >
>
>
>
>

From: kuldeep lakhani (2011-08-05 01:40)

We have similar issue of Websphere hanging and breaking down several times in a day.
Sometimes we found even log file not updating when server breaks, though other times we noticed different errors like - "..Out of memory..", (This is most of the times)- "..unable to create thread..",- "CPU starvation"
We have two servers, each with 8GB RAM; Web Sphere - Maximo 6.2 - on Windows 2003. DB Server is having Oracle 11g.
So far, no luck to find out the cause. Any idea would be appreciated please.
Thanks & Regards...Kuldeep S. LakhaniMobile: +971 56 6435 365 Never Postpone The Joy !!!
--- On Thu, 7/28/11, ssc.susser <halexander@susser.com> wrote:
From: ssc.susser <halexander@susser.com>
Subject: Re: [MAXIMO List] Wepsphere - MxServer Slow performance
To: MAXIMO@yahoogroups.com
Date: Thursday, July 28, 2011, 12:18 PM

We were having a huge problem with MxServer locking up. Setting up a cluster really did wonders to increase the up time, but it turns out our problem was with an interaction node in one of our workflows. It had worked previously but when we applied an update the action in the interaction node "routewf" caused an infinite loop. You might want to look into something like that. We ended up tracking down the problem by looking at the wfinstance table, grouping by user, and count() the number of wfinstance records and seeing the massive task calls for that one user.
The pattern for our freezes was very erratic--never happened at the same time of day and not always on the same cluster member.
Are you noticing massive CPU spikes or are you getting OOM errors or both?
--- In MAXIMO@yahoogroups.com, Brian Scott <bpscotty@...> wrote:
>
> If you have 3 app servers in your was node the box it runs on will need at least 5GB ram.
> Set the heap sizes for DMgr to 256 512 and each app server to 512 1560
>
> Are you running it all in one windows server?
>
> Probably best to have the db2 on it's own box with 4GB min ram
>
> Brian Scott
> Service Management Connect
>
> Sent from my iPad
>
> On 26 Jul 2011, at 23:29, shah faisal <dba_develper@...> wrote:
>
> > Dear All, Good Morning
> >
> > DB: DB2
> > OS: Windows 2003 sp2
> > Maximo 7.1
> >
> > My websphere 'MxServer' is extreemly slow and more than 4 times a day I have to restart the server. Kindly help me dig out the problem with solution.
> > I have also other application servers such as MxCron,MxTest.
> >
> > I do not know how to resolve that issue? What steps need to be taken?
> >
> > I need your urgent help in detail.
> >
> > Regards
> > Shah
> >
>
>
>
>

From: Chris Lawless (2011-08-05 10:37)

Try enabling verbosegc logging so you can see what was happening with the java heap leading up to the crash. The main cause of CPU starvation is a JVM frantically trying to free up heap space until it destroys itself. It is likely that you have a process using an abnormally large amount of heap or some code with a memory leak. My guess would be hat leading up to the crash here is no available heap and pause times increase until the CPU cannot keep p and crashes.
If this is the case the next step is to figure out The cause, capturing the MAXSESSIONcontent for the JVM should help narrow down who was connected so you can try to determine what they were doing. Have you enabled the standard performance monitors for MBOSet counts and SQL timings?
Chris.
On Aug 5, 2011, at 9:40, kuldeep lakhani <kuldeep_lakhani2000@yahoo.com> wrote:
> We have similar issue of Websphere hanging and breaking down several times in a day.
> Sometimes we found even log file not updating when server breaks, though other times we noticed different errors like - "..Out of memory..", (This is most of the times)- "..unable to create thread..",- "CPU starvation"
> We have two servers, each with 8GB RAM; Web Sphere - Maximo 6.2 - on Windows 2003. DB Server is having Oracle 11g.
> So far, no luck to find out the cause. Any idea would be appreciated please.
> Thanks & Regards...Kuldeep S. LakhaniMobile: +971 56 6435 365 Never Postpone The Joy !!!
>
> --- On Thu, 7/28/11, ssc.susser <halexander@susser.com> wrote:
>
> From: ssc.susser <halexander@susser.com>
> Subject: Re: [MAXIMO List] Wepsphere - MxServer Slow performance
> To: MAXIMO@yahoogroups.com
> Date: Thursday, July 28, 2011, 12:18 PM
>
>
>
> We were having a huge problem with MxServer locking up. Setting up a cluster really did wonders to increase the up time, but it turns out our problem was with an interaction node in one of our workflows. It had worked previously but when we applied an update the action in the interaction node "routewf" caused an infinite loop. You might want to look into something like that. We ended up tracking down the problem by looking at the wfinstance table, grouping by user, and count() the number of wfinstance records and seeing the massive task calls for that one user.
>
> The pattern for our freezes was very erratic--never happened at the same time of day and not always on the same cluster member.
>
> Are you noticing massive CPU spikes or are you getting OOM errors or both?
>
> --- In MAXIMO@yahoogroups.com, Brian Scott <bpscotty@...> wrote:
>
> >
>
> > If you have 3 app servers in your was node the box it runs on will need at least 5GB ram.
>
> > Set the heap sizes for DMgr to 256 512 and each app server to 512 1560
>
> >
>
> > Are you running it all in one windows server?
>
> >
>
> > Probably best to have the db2 on it's own box with 4GB min ram
>
> >
>
> > Brian Scott
>
> > Service Management Connect
>
> >
>
> > Sent from my iPad
>
> >
>
> > On 26 Jul 2011, at 23:29, shah faisal <dba_develper@...> wrote:
>
> >
>
> > > Dear All, Good Morning
>
> > >
>
> > > DB: DB2
>
> > > OS: Windows 2003 sp2
>
> > > Maximo 7.1
>
> > >
>
> > > My websphere 'MxServer' is extreemly slow and more than 4 times a day I have to restart the server. Kindly help me dig out the problem with solution.
>
> > > I have also other application servers such as MxCron,MxTest.
>
> > >
>
> > > I do not know how to resolve that issue? What steps need to be taken?
>
> > >
>
> > > I need your urgent help in detail.
>
> > >
>
> > > Regards
>
> > > Shah
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>

From: kuldeep lakhani (2011-08-12 21:30)

Hi,We increased INITIAL HEAP SIZE value in JVM, from 512 to 1024.While we are still monitoring the performance, above is working way beyond expectations so far reducing our crashing to 10-20% only.
Thanks for all you guys there with advise. Regards...Kuldeep
--- On Fri, 8/5/11, Chris Lawless <lawlessc@gmail.com> wrote:
From: Chris Lawless <lawlessc@gmail.com>
Subject: Re: [MAXIMO List] Wepsphere - MxServer Slow performance
To: "MAXIMO@yahoogroups.com" <MAXIMO@yahoogroups.com>
Date: Friday, August 5, 2011, 4:37 AM

Try enabling verbosegc logging so you can see what was happening with the java heap leading up to the crash. The main cause of CPU starvation is a JVM frantically trying to free up heap space until it destroys itself. It is likely that you have a process using an abnormally large amount of heap or some code with a memory leak. My guess would be hat leading up to the crash here is no available heap and pause times increase until the CPU cannot keep p and crashes.
If this is the case the next step is to figure out The cause, capturing the MAXSESSIONcontent for the JVM should help narrow down who was connected so you can try to determine what they were doing. Have you enabled the standard performance monitors for MBOSet counts and SQL timings?
Chris.
On Aug 5, 2011, at 9:40, kuldeep lakhani <kuldeep_lakhani2000@yahoo.com> wrote:
> We have similar issue of Websphere hanging and breaking down several times in a day.
> Sometimes we found even log file not updating when server breaks, though other times we noticed different errors like - "..Out of memory..", (This is most of the times)- "..unable to create thread..",- "CPU starvation"
> We have two servers, each with 8GB RAM; Web Sphere - Maximo 6.2 - on Windows 2003. DB Server is having Oracle 11g.
> So far, no luck to find out the cause. Any idea would be appreciated please.
> Thanks & Regards...Kuldeep S. LakhaniMobile: +971 56 6435 365 Never Postpone The Joy !!!
>
> --- On Thu, 7/28/11, ssc.susser <halexander@susser.com> wrote:
>
> From: ssc.susser <halexander@susser.com>
> Subject: Re: [MAXIMO List] Wepsphere - MxServer Slow performance
> To: MAXIMO@yahoogroups.com
> Date: Thursday, July 28, 2011, 12:18 PM
>
>
>
> We were having a huge problem with MxServer locking up. Setting up a cluster really did wonders to increase the up time, but it turns out our problem was with an interaction node in one of our workflows. It had worked previously but when we applied an update the action in the interaction node "routewf" caused an infinite loop. You might want to look into something like that. We ended up tracking down the problem by looking at the wfinstance table, grouping by user, and count() the number of wfinstance records and seeing the massive task calls for that one user.
>
> The pattern for our freezes was very erratic--never happened at the same time of day and not always on the same cluster member.
>
> Are you noticing massive CPU spikes or are you getting OOM errors or both?
>
> --- In MAXIMO@yahoogroups.com, Brian Scott <bpscotty@...> wrote:
>
> >
>
> > If you have 3 app servers in your was node the box it runs on will need at least 5GB ram.
>
> > Set the heap sizes for DMgr to 256 512 and each app server to 512 1560
>
> >
>
> > Are you running it all in one windows server?
>
> >
>
> > Probably best to have the db2 on it's own box with 4GB min ram
>
> >
>
> > Brian Scott
>
> > Service Management Connect
>
> >
>
> > Sent from my iPad
>
> >
>
> > On 26 Jul 2011, at 23:29, shah faisal <dba_develper@...> wrote:
>
> >
>
> > > Dear All, Good Morning
>
> > >
>
> > > DB: DB2
>
> > > OS: Windows 2003 sp2
>
> > > Maximo 7.1
>
> > >
>
> > > My websphere 'MxServer' is extreemly slow and more than 4 times a day I have to restart the server. Kindly help me dig out the problem with solution.
>
> > > I have also other application servers such as MxCron,MxTest.
>
> > >
>
> > > I do not know how to resolve that issue? What steps need to be taken?
>
> > >
>
> > > I need your urgent help in detail.
>
> > >
>
> > > Regards
>
> > > Shah
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>