dominoExperts.com - Powered by Domino 8.5.2 Domino Accelerator Pack
- DAP know which URLs slow down your site. Do you?
Lotus Triple Search DominoExperts + Blogs + R8 forum
dominoExperts.com -> Domino Administration

 Troubleshooting a server with poor performance


Tomas NielsenPost date: 2007-09-07 18:25

A customer has a server that fills up the CPU almost every morning causing it to stop answering incoming HTTP traffic.

I am not the one running the server so I have limited access. But as always seeing a Domino server in pain I want to give a helping hand.

I have spotted a few things so far. Could you guys help me out with pointing out if it is normal?

- 4 agent managers running.
- A scheduled program that runs: nServer -c "dbCache flush" every 30 minutes(!)
- Compact is run each morning and evening
- 3 scheduled programs that run updall on 3 different database every 30 min
- >1000 agents that are scheduled on the server
- Lots of Notes client users
- Lots of web traffic

Also if you could give some pointers to what quick fixes can be applied - short term - to get the machine into better shape.


Niklas WallerPost date: 2007-09-10 21:15

Hi,
I can understand the compassion you feel for the server ;-) It is certainly busy!

First of all, a 1000 scheduled agents are a lot of agents and combined with a large number of databases, some large and heavily used, with many notes and web users; this could itself lead to performance problems. Many notes users with large inbox folders could for example contribute to bad performance as a large number of concurrent web users could.

In long-term I would consider clustering this server on at least on more server depending on how many databases and users it has. I would also start looking at the code if there are a lot of lookups to different data sources etc. Also go through rules and policies like mail quotas etc.

In short-term I would limit the number of scheduled programs. In my opinion there are several things that are overdone here. I use compact with file size reduction (-B) once a week and simple in-place compaction once a day perhaps if even necessary. I would recommend running Updall once a week as well and not every 30 minutes.
I would turn off the dbCache flush scheduled program and instead if needed increase the NSF_Buffer_Pool_Size in notes.ini. If there is enough space in the cache it would instead help increasing performace. On the other hand if the cache isn't big enough it could slow down performance noticeable.

Thomas Adrian posted a top 10 list the other day of for notes admins which is good reading. Here's the english version he found.

Anyway, a little feedback!
/ Niklas


Tomas NielsenPost date: 2007-09-12 23:09

Thanks for the feedback!

About the scheduled "dbcache flush". Is that something that is normally done? I have never seen it before.
I use it to get domino to release files if I want to move them but I see no logical explanation to used it on a schedule.

I agree that the compact jobs are overdone.

And the agents... Fortunately I am only responsible for 7 of the 1000 agents. 


Niklas WallerPost date: 2007-09-13 13:33

Hi,
I have never seen that before either, especially not with that interval. The meaning with the command is to release databases from the cache before maintenance, as you described. I don't think it was intended to schedule though. The reason it is on that server is maybe a misunderstanding that there is a need to flush it before doing the updall, i.e. maintenance (which also was scheduled to run every 30 minutes).

Lucky you! Better chance of doing a good job with only 7 seven agents


Bryan KuhnPost date: 2007-09-13 16:00

Tomas,

In terms of the agents, if there are any LotusScript agents try to avoid having Dim statements embedded within any form of Loop.  I've seen this cause server issues.  More specifically I've seen this cause a server crash as a result of running out of backend memory.


Tomas NielsenPost date: 2007-09-15 10:41

There is probably some LotusScript agents that allocate memory in there too. We have little knowledge of what all those agents are doing at the moment.

Some progress have been made. We have identified at least one agent that runs for 30 minutes (max runtime) and then times out without being finished. Can not be good.

Fortunately our 7 agents are still behaving.




RSS feed
Subscribe to Forum

Share this page

Top posters
Tomas Nielsen212
Joacim Boive27
Fredrik Stöckel27
Danne14
Niklas Waller13
Kenneth Haggman11
Bryan Kuhn10
Daniel Lehtihet9
Jonas Israelsson8
dm997