Continuously increasing Apache memory on Windows

[Nitin]

Windchill 10.1 M030 configured with 7 foreground and 3 BGMS.

Apache memory (child httpds) increases to 1.1 GB in 2 weeks. As a result we have to restart PRODUCTION environment every two weeks and some times even before that.

We are working with PTC TS to resolve this. Nothing wrong identified with config till yet.

It's not reproducible in TEST environment (with less user load), and this makes more difficult to debug (since TEST environment is the most closest in SW/HW config). I tried simulating on DEV (without https) and the way I can reproduce it is using Apache Benchmark tool.

Will be interested to know, if any one here has observed this issue in their environment, if yes - what you did to fix it?

Will also appreciate if I can get get some leads to debug it?

Thanks,

Nitin

[grocha]

You should check the architecture of your environment including your
integrations, in a scenario were you have plenty of concurrent integration
jobs (via webserver for example), the best option would be to implement some
load balance directing all the integrations to a second server of the
cluster.

You are probably aware of your environment but it's just a thought that
occurred me now.





Best of luck

[Nitin]

We are running Windchill in Veritas Cluster with one active and one passive node. Same is true for Oracle.

The failover happens only in cases when the service level resource becomes non-responsive.
After failover, Windchill is able to switch to other cluster node, and this is how we maintain business continuity as of now.

Running Windchill in cluster - That means additional resources and will take long time.
On a shorter term and to build upon that, I need to identify first if it's the load which is causing this issue. Ho would you do that?

[Nitin]

Thanks.

We are running Windchill in Veritas Cluster with one active and one passive node. Same is true for Oracle.

The failover happens only in cases when the service level resource becomes non-responsive.
After failover, Windchill is able to switch to other cluster node, and this is how we maintain business continuity as of now.

Running Windchill in cluster - That means additional resources and will take long time.
On a shorter term and to build upon that, I need to identify first if it's the load which is causing this issue.

How would you do that?

-Nitin

In Reply to Guilherme Rocha:

You should check the architecture of your environment including your
integrations, in a scenario were you have plenty of concurrent integration
jobs (via webserver for example), the best option would be to implement some
load balance directing all the integrations to a second server of the
cluster.

You are probably aware of your environment but it's just a thought that
occurred me now.





Best of luck

[Nitin]

Some additional testing results:

Apache 2.2.22 running on two different machines on Windows Server 2008 SP1.

Machine 1:
Intel(R) Xeon(R) CPU E5-2609 @2.40 GHz (2 processors)
RAM: 128 GB
PRODUCTION Machine

Machine 2:
Intel(R) Xeon(R) CPU E5-2609 @2.40 GHz (2 processors)
RAM: 32 GB
TEST Machine

Machine 3:
AMD Opteron(tm) Processor 6176 SE 2.29 GHz (4 processors)
RAM: 12 GB
DEVELOPMENT Machine



On Machine 1, we observe that the size of the httpd.exe process increases to 1.2 Gb.

On Machine 2 and Machine 3: we run the Apache Benchmark tool to simulate the load similar to PRODUCTION

abs -n 500000 -c 50 -A username:password https://<domain>/application

Using the tool we can see httpd.exe (child process) memory reaching to 1.2 GB on Machine 3, but not on Machine 2.


All the three machines have the same apache configuration.
KeepAliveTimeout 60
SendBufferSize 16384
Timeout 900
ThreadsPerChild 600

Forgot to mention -

Machine 1 and Machine 2 is running with SSL. (https)

Machine 3 NOT (https).

Any help or pointers would be appreciated.

Thanks,
Nitin