Level: 200
 

Threshold exceeds

Bomb

If you are going through the Sitecore log of your installation, you may have noticed some log entries reporting of exceeded thresholds. The message looks something like this:

 

2188 20:59:06 WARN  Timing threshold exceeded for web page. Milliseconds: 38622,88. Threshold: 1000. Page URL: /Default.aspx

 

These are very common, but should not appear in larger numbers, as this might mean that you have some performing issues.
This article describes why they appear and how to troubleshoot them.

Written by: Jens Mikkelsen
Fri, Jun 26 2009

What is the threshold exceeds?

There are three different thresholds in Sitecore: The timing, memory and item thresholds. They all work by using a counter. The counter is initialized when an end user requests a page by entering a URL. More specific the counter is initialized in the httpRequestBegin pipeline in the StartMeasurement processor. When the request ends, the counter is measured and if the number exceeds a threshold set in the web.config, it is logged in the Sitecore log. The log entry includes an indication of what type of exceed it is (time, memory or item), how much the counter value is, the threshold and the URL which were requested.

 

The timing threshold

The timing threshold indicates if a request takes too long. A timer is started upon request and when the request ends, the time is measured. By default the threshold is set to 1000ms which corresponds to 1 second. So when you see a log entry like this:

 

2188 20:59:06 WARN  Timing threshold exceeded for web page. Milliseconds: 38622,88. Threshold: 1000. Page URL: /Default.aspx

 

It indicates that someone has requested the URL /default.aspx and Sitecore spent over 38 seconds processing the request. As a system administrator I know this sounds a bit alarming, and it may be. In this incident I know it is because the .Net process restarted.

What you need to be aware of, is that the timer is not CPU time, but total time for the request. This means that the reason for the long response time, may be caused by other things than the request itself. So you cannot use a single entry to point out anything. What you can do however, is getting statistics over timing threshold exceeds. By parsing the log you should be able to identify if a certain URL repeatedly results in an exceed. If this is the case, the page requested may have a performance issue.

 

Timing threshold exceeds can be caused by a lot of things. Generally you need to consider if threshold exceeds are revolving around a single or few URL’s or if it is spread out on a lot of URL’s. If the entries report the same URL a lot, this often mean that you have a rendering or sublayout on the page, which has some bad performing logic. If the URL’s are spread out it could be that you’re servers aren’t able to keep up with requests or you’re network between your SQL server and web server may be having problems. It could be a lot of things, but it is probably more of a general performance issue.

 

The memory threshold

The memory threshold works similar to the timing threshold. When a request is handled by Sitecore the StartMeasurement processor looks at the total amount of memory currently allocated. When the request ends the delta in total memory allocated is measured. If the delta is larger than the threshold specified in the web.config, it is reported with a log entry like this:

 

2188 20:59:06 WARN  Memory threshold exceeded for web page. Memory used: 21399KB. Threshold: 10000KB. Page URL: /Default.aspx

 

This entry says that more than 21mb of memory has been allocated during the time this request has taken.
Again you need to be careful how you interpret this information. First of all you can’t be sure that it is the given request, which allocated the memory. Also the longer the request has taken, the longer is the period for measuring the delta and hence more natural with the increase. The cause of the memory exceed is therefore difficult to determine and I would generally recommend using the standard .net performance counters instead.

 

The item threshold

The item threshold is probably the most important counter, as it gives some really good indications of what is slowing your site down. You can’t find this type of information in any other .net debugging tool. The counter works by initiating a counter in the beginning of the request, indicating how many items in Sitecore has been accessed during the period of the request.

In the same way as the other counters it works globally and isn’t bound to the single request. So you can’t use a single log entry to analyze anything. Instead you should draw statistics and you will probably see a clear indication which pages iterates over a large amount of items. This is usually the number 1 reason, why a site won’t perform. So if you see a lot of log entries looking like this:

 

2912 21:10:28 WARN  Item threshold exceeded for web page. Items accessed: 2633. Threshold: 1000. Page URL: /Default.aspx

 

It is probably caused by a rendering using a //item or descendant or similar. This can completely kill the performance of your site, and should be handled. The solution is often to use an index rather then iterating over a complete tree of items.

 

Should I adjust the default thresholds?

Often when I visit a client with performance problems, they have either increased the thresholds dramatically or completely removed them. They claim that the massive amount of log entries was slowing down the site a lot. Now this is a really bad argument. Not only is it fixing the symptoms and not the problem, you also disable a future consultant from using the information for debugging. My experiences are that the default threshold values are very accurate. If you see too many entries, it is probably because, you have a rendering or sublayout which isn’t performing well enough for production. So my advice is… Don’t adjust the settings.
 

 

 

Please rate this article


7 rates / 4,14 avg.

  • About the author:

    Jens Mikkelsen

    Jens Mikkelsen is a partner at Inmento Solutions a Sitecore consulting firm. He works as a Sitecore specialist and consulting helping clients architect and build quality Sitecore solutions using the newest modules and tools. 

    Further he has been deeply envolved in various complex solutions and has built up a strong knowledge of Sitecore architecture and best practices. He has especially focused on and is specialized in debugging and analyzing Sitecore solutions.

     

    Jens is very interested in the technical mechanisms in the new marketing products such as Sitecore DMS and Sitecore ECM.

    My Sitecore Freelance CV

21 responses to "Threshold exceeds"

Hi Jens,

Thanks for the insight. I am trying to track down some performance issues right now and your post is very relevant. Is it sitecore 6 specific? I read in your post on SDN (http://sdn5.sitecore.net/forum//ShowPost.aspx?PostID=9686) that the thresholds are not per request but an aggregate over the timeframe in question. Is this no longer true in sitecore 6? We didn't have any particular performance issues on v5.3 per se but have seen a surge in performance-related problems since upgrading to sitecore 6 last weekend. My site gets a solid amount of traffic so I am not sure how valid these thresholds are for me. Do you have any tricks for tracking down performance issues with specific renderings or sublayouts you would be willing to share?

Thanks in advance,
will
Posted: Sunday, August 09, 2009 5:33 AM
Hi Will.
This article is not Sitecore 6 specific.
Which of the thresholds are your site exceeding?
Is it a problem with all of your pages, or only some of them?
you should be able to get this information in the Sitecore log.
Have you tried running the Sitecore debug functionallity? It is located in the
Sitecore menu.
By running the debugger you are able to profile each rendering that is used on the page by hovering the green arrows.
It will also give you a top three over the renderings that uses most time and reads most items.
-Jimmi
Posted: Sunday, August 09, 2009 10:52 PM
Hmmm... Sitecore 6.0 is to my experience better performing than Sitecore 5.3. So unless you upgraded to Sitecore 6.1 it shouldn't be the product Sitecore which gives you an headache... I find that the IIS log in combination with a log parser is quite helpfull when dealing with performance issues. You can parse logs to see which URL's have long response times etc. Try and google IIS logparsing and you should be well of. Further to what Jimmi said the /sitecore/admin/stats.aspx can be quite helpfull.
Posted: Sunday, August 09, 2009 11:25 PM
Jens, good point about the iis log parsing, I will check that out. we did not have this performance issue prior to our upgrade which is why i was surprised and thinking it may be v6-specific. Our in-house testing showed that the performance was indeed better than in 5.3 but now under full load the sitecore w3 worker process just eats up memory until it croaks.

Jimmi, I was thinking the thresholds may hold some clue to the performance situation but even loading a media library item exceeds the default item threshold so there is just too much data to see a pattern. This is true for almost every item. I upped all three thresholds by a factor of 10. I did the same adjustment for 5.3 when we first launched and it seemed to help performance overall. We used the debugger when we first started coding but for some reason have gotten away from it recently. I will fire it up again and see what it can tell me.

Thanks for the input. If I discover what is going wrong, I'll post back. Right now I just have the app pool recycling every 8 hours which so far has seemed to keep things under control as far as memory usage and crashed but performance is still not quite there.
Posted: Tuesday, August 11, 2009 11:34 PM
I get these Item threshold warnings from every page in the sample site that ships with the Developer version of Sitecore. This seems somewhat worrying... The sample pages are dead simple, why is this happening?
Posted: Tuesday, September 29, 2009 8:41 PM
I am not familiar with the code in the starter kit, so I don't know exactly what might be causing it. Normally when I see a lot of item threshold exceeds, I look for an XSL or ascx which iterates through all items with "//item" or "descendants-or-self".
Posted: Tuesday, September 29, 2009 9:43 PM
Note that it is normal to have threshold exceeds in the log. If you don't experiene performance issues, it'll probably not be a problem. Also note that the starter kit probably isn't tuned for high scale and lots of content.
Posted: Tuesday, September 29, 2009 9:46 PM
Bryan,

My experience is that those threshold numbers are fairly bogus (sorry, Jens) since they are not based on a singleton type measurement but the aggregate of concurrent accesses. That seems to be why I constantly see threshold warnings even for items in the media library which would not seem to be involved in any xpath type heavy lifting. I have been much more successful pinpointing poor performance spots by using other tools like the Sitecore debugger, yslow and http watch then re-factoring renderings or re-writing long running renderings as sublayouts, especially since we moved to Sitecore 6.0. Overall I ignore them for the most part because there is too much noise. Just my opinion, though.

- will
Posted: Tuesday, September 29, 2009 11:29 PM
No worries Bryan. ;) I agree that you cannot use a single entry to anything. However I have really good experience using them statistically. Working in Pentia Professionel Services I am often called out to debug a solution and identify why the site isn't performing. My greatest tool in diagnosting a solution is the thresholds. I have a log parser running through the log file, which collects how many thresholds exceeds has occured, and which URLs most often exceeds the threshold. In 90% of the time this pinpoints me to which page or pagetype is causing the majority of the problems in just 5 minutes.
Posted: Tuesday, September 29, 2009 11:46 PM
Oh and I meant "No worries, Will"
Posted: Tuesday, September 29, 2009 11:47 PM
Hi Jens

We have a "many items site" (5.3.1) with performance problems and have quite a lot of the threshold mentioned in your article. (all 3 actually)
the item blog say sometimes more than 200000 items accessed- Guess thats far to may ;-)
So thought about going through the renderrings for optimizing. But..

the performance issue is also heavy in the sitecore backend. which should not relate to the frontend renderrings? - Do you think it could be the frontend renderrings slowíng down performance of the server in general?

Nicolai
Posted: Tuesday, March 09, 2010 9:25 AM
Hi Nicolai,

If the backend and frontend is hosted on the same server, it is very likely that the frontend can slow down the backend.

I would start by looking through all your presentations to see if the descendants-or-self or "//item" is used. If this is the case, you should try and get around it.

Please let me know if you have more questions.

Cheers
Jens
Posted: Tuesday, March 09, 2010 9:32 AM
Hi Jens

Thx for answering!

I have now spent some time trying to solve the performance issue on the site I mentioned above.I havent solved it 100% but heres where I am:
It seems like that the problem with the backend performance is related to one specific template.
In this case its called "abstract template". A lot of other templates a inheriting from this one, since its the one containing the most used fields like title, content etc.

I tracked down that all items based on a template using this "abstract template" also are the ones performing badly. (45 sec. for load in backend) All items not based on this template is working ok. (loads in approx 1. sec.)
I made a fast test, where I tried making a new template with fields with same names as in the "abstract template") - I then used the "change template" function for an item. And after changing template the item is not performing bad any more.
Havent assigned layout and all fields to the new template though, but theres quite a heavy indication that its the "abstract template" where the bug is. (Seems like deadlock or that its searching for items that s not there or something)

My problem is that there is around 90.000 items in the solution, and a lot of the items is using the "abstract template" - a change template solution would really be though.
Would be better solving the issue in the "corrupted" "abstract template". But can find any bugs, and do not have any tools for identifying it.
Have you (or other reading this) expericenced a problem alike? Or do you have any idea on how to get to next step? ;-)
Posted: Monday, March 29, 2010 3:21 PM
Hi Jens

Thx for answering!

I have now spent some time trying to solve the performance issue on the site I mentioned above.I havent solved it 100% but heres where I am:
It seems like that the problem with the backend performance is related to one specific template.
In this case its called "abstract template". A lot of other templates a inheriting from this one, since its the one containing the most used fields like title, content etc.

I tracked down that all items based on a template using this "abstract template" also are the ones performing badly. (45 sec. for load in backend) All items not based on this template is working ok. (loads in approx 1. sec.)
I made a fast test, where I tried making a new template with fields with same names as in the "abstract template") - I then used the "change template" function for an item. And after changing template the item is not performing bad any more.
Havent assigned layout and all fields to the new template though, but theres quite a heavy indication that its the "abstract template" where the bug is. (Seems like deadlock or that its searching for items that s not there or something)

My problem is that there is around 90.000 items in the solution, and a lot of the items is using the "abstract template" - a change template solution would really be though.
Would be better solving the issue in the "corrupted" "abstract template". But can find any bugs, and do not have any tools for identifying it.
Have you (or other reading this) expericenced a problem alike? Or do you have any idea on how to get to next step? ;-)
Posted: Monday, March 29, 2010 3:27 PM
Hi Jens

Thx for answering!

I have now spent some time trying to solve the performance issue on the site I mentioned above.I havent solved it 100% but heres where I am:
It seems like that the problem with the backend performance is related to one specific template.
In this case its called "abstract template". A lot of other templates a inheriting from this one, since its the one containing the most used fields like title, content etc.

I tracked down that all items based on a template using this "abstract template" also are the ones performing badly. (45 sec. for load in backend) All items not based on this template is working ok. (loads in approx 1. sec.)
I made a fast test, where I tried making a new template with fields with same names as in the "abstract template") - I then used the "change template" function for an item. And after changing template the item is not performing bad any more.
Havent assigned layout and all fields to the new template though, but theres quite a heavy indication that its the "abstract template" where the bug is. (Seems like deadlock or that its searching for items that s not there or something)

My problem is that there is around 90.000 items in the solution, and a lot of the items is using the "abstract template" - a change template solution would really be though.
Would be better solving the issue in the "corrupted" "abstract template". But can find any bugs, and do not have any tools for identifying it.
Have you (or other reading this) expericenced a problem alike? Or do you have any idea on how to get to next step? ;-)
Posted: Monday, March 29, 2010 3:53 PM
Hi Nicolai,

Hmmm. Sounds weird. Is the "abstract template" by accident inheriting from itself? Is there any fields on "abstract template", which is a multilist, which has to many items in it?

Cheers
Jens
Posted: Sunday, April 04, 2010 2:43 PM
Hi Jens,

Great post! Even it's some time now since you posted it still helped me a lot.
I have one question... You say "The solution is often to use an index rather then iterating over a complete tree of items."
What are you referring at with "index"? Is it a Lucene index? How should it be created?

Thanks,
Albert
Posted: Thursday, September 29, 2011 1:16 PM
Hi Albert,

Yes... I refer to a Lucene index. You can see how to create an index here: http://learnsitecore.cmsuniverse.net/en/Developers/Articles/2009/06/LuceneQuery1.aspx

Please note that the search API recently was updated by Sitecore (6.5 I think), so some of the information in the above article may be outdated.
Posted: Thursday, September 29, 2011 3:08 PM
Hi Jens,

Great post! Even it's some time now since you posted it still helped me a lot.
I have one question... You say "The solution is often to use an index rather then iterating over a complete tree of items."
What are you referring at with "index"? Is it a Lucene index? How should it be created?

Thanks,
Albert
Posted: Thursday, September 29, 2011 3:19 PM
Hi Jens,

Thanks a lot for your reply... I will look into the link you shared.

Cheers,
Albert
Posted: Thursday, September 29, 2011 3:24 PM
Thresholds should be disabled as they only make sense in an environment with only 1 visitor per page, as the counters are aggregated for all users per page. So if 100 user access the same page and it accesses 10 items and the threshold is 900 it will write the warning in the log.

So these numbers should only be used in a testing environment, they are useless in production and should be disabled.
https://sdn.sitecore.net/faq/troubleshooting/item%20threshold%20exceeded%20for%20web%20page.aspx
Posted: Thursday, November 05, 2015 12:48 PM

Leave a reply


Notify me of follow-up comments via email.
 
 
#nbsp;