Directory Programming .NET

Active Directory and ADAM programming support for .NET developers
Welcome to Directory Programming .NET Sign in | Join | Help
in Search

MaxQueryDuration timeout of 2 minutes and searching

Last post 06-25-2008, 3:16 PM by dunnry. 13 replies.
Sort Posts: Previous Next
  •  06-23-2008, 2:09 PM 4004

    MaxQueryDuration timeout of 2 minutes and searching

    If I do a DirectorySearcher.FindAll() and then loop through the SearchResultsCollection, where does the default AD server timeout (MaxQueryDuration) apply?

    (a) Just to the FindAll() call
    (b) To the entire process of looping through the SearchResult objects

    Thanks,

    Sandy
  •  06-23-2008, 2:50 PM 4006 in reply to 4004

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Do you have the book?  I know we covered this in Chapter 5 on timeouts.  This particular timeout will only apply to non-paged queries.  By default it is 2 min, but it can be set to anything by your admin.  As soon as you apply paging however, this one is ignored.

    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
  •  06-23-2008, 3:40 PM 4008 in reply to 4006

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Hi Ryan,

    Yes, I do have your book.  I just pulled it out again. I could have sworn that one of the Microsoft Tech Support guys said the 2-minute limit still applied to the whole search.

    Anyway, I am using paged searching and I have several customers that are timing out after 2 minutes.  If I understand your book correctly, MaxQueryDuration is going to apply on a page-by-page basis.  Here are the settings I am using:

    Filter -  (&(objectClass=User)(objectCategory=Person)(!userAccountControl:1.2.840.113556.1.4.803:=2))

    We tried taking out (objectClass=User), but they still time out

    ClientTimeout - not set
    MaxPageSize - 1000 (we also tried 100, still time out)
    ServerPageTimeLimit  - 120 - (will be MaxQueryDuration anyway)
    ServerTimeLimit - not set
    SizeLimit - not set
    Sort - none
    Referral Chasing - None
    SearchScope - Subtree

    We are using .NET 2.0.  Is there any service pack that customers could be missing that fixed something with AD searching?

    I am completely out of ideas.  I have been trying to fix this problem for months. I would really appreciate your help.
  •  06-23-2008, 5:07 PM 4009 in reply to 4008

    Re: MaxQueryDuration timeout of 2 minutes and searching

    According to the book, the 2 min limit applies per page when using paging as you say.  I recall researching this a lot at the time of writing, so I think it is accurate.  It definitely does not apply to the whole paged search as you would never be able to have a search longer than 2 minutes.  As long as you are paging, this limit should never really affect you.

    How many users are we talking about here?  The particular filter you are running is a little slow just because of the bitwise filter, but it should still return in under 2 minutes in most domains I have seen including some very large ones.

    Also, when you say that something is timing out, can you elaborate on the error?  For instance, if you are timing out on the query from a console app, this is more unexpected than say an ASP.NET web app...


    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
  •  06-23-2008, 5:18 PM 4011 in reply to 4009

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Also, what happens if the ServerPageTimeLimit is set to something less than MaxQueryDuration instead of the same number?

  •  06-23-2008, 5:20 PM 4012 in reply to 4009

    Re: MaxQueryDuration timeout of 2 minutes and searching

    The current customer that is timing out says he has 4100 users. He says about 700 should be coming back with the search filter he's using:

    (&(objectClass=user)(objectCategory=Person)(!userAccountControl:1.2.840.113556.1.4.803:=2)(employeeID=*))

    He says using employeeID=* filters out records that do not have an employee ID.

    I had him lower the MaxPageSize to 1 and he still times out.  So, my current theory is either (1) the MaxQueryDuration is applying to the whole query or (2) somehow, the paged search is not working.  I stepped the debugger and I see the MaxPageSize and ServerPageTimeLimit values set. Is there some other property I need to set?

    Thanks!
  •  06-23-2008, 5:22 PM 4013 in reply to 4012

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Oh, I forgot to mention that all of this is happening in our WinForms application. We run 3-tier using .NET remoting.
  •  06-23-2008, 5:23 PM 4014 in reply to 4012

    Re: MaxQueryDuration timeout of 2 minutes and searching

    You don't want to mess with the MaxPageSize - setting it to 1 means you will saturate your network with lots and lots of calls.  Something else is the issue here... 4100 users is not enough (not remotely enough) to make this query slow.

    Can I see the code?  Is this an ASP.NET app?

    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
  •  06-23-2008, 5:36 PM 4015 in reply to 4014

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Don't worry. We only set the page size to 1 to prove that the query would still time out. We wouldn't really do that.  :-)

    There is no way I could send a piece of my WinForms app that would be meaningful. We are metadata driven and very OO.  Let me see if I can write a test app and have my customer run it to make sure it times out for him.  Then, I'll post it.

    Thanks!
  •  06-23-2008, 5:49 PM 4016 in reply to 4015

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Yep, my next suggestion was going to be running this in a simple console app or even something like SnippetCompiler or LINQPad to prototype the code.

    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
  •  06-25-2008, 1:38 PM 4031 in reply to 4016

    Re: MaxQueryDuration timeout of 2 minutes and searching

    Here is a sample that shows how I've written my code.  To run it you pass the start search path, the user ID and the password.  For example:

    ActiveDirectoryTimeout "LDAP://192.168.0.156/dc=Sandy,dc=home,dc=com" Administrator password


    using System;
    using System.Collections;
    using System.DirectoryServices;
    using System.Text;

    namespace ActiveDirectoryTimeout
    {
        class Program
        {
            static void Main(string[] args)
            {
                if (args.Length < 3)
                    Console.WriteLine("Usage is: ActiveDirectoryTimeout <LDAP start path> <userId> <password>");

                Program pgm = new Program();
                pgm.ImportUsersFromLDAP(args[0] as string, args[1] as string, args[2] as string);
            }

            public void ImportUsersFromLDAP(string strLDAPPath, string strUserId, string strPwd)
            {
                ArrayList alAttributesToLoad = new ArrayList();
                int nCount = 0;
                ResultPropertyValueCollection col = null;
                string strItemPath = "";
                string strFilter = "(&(objectClass=user)(objectCategory=Person)(!userAccountControl:1.2.840.113556.1.4.803:=2)(EmployeeID=*))";
                int nMaxPageSize = 1000;
                int nPageTimeLimitInSecs = 120; //max we can set since MaxQueryDuration will kick in at 120 seconds

                try
                {
                    //Add attributes to load from User object
                    alAttributesToLoad.Add("department");
                    alAttributesToLoad.Add("distinquishedName");
                    alAttributesToLoad.Add("employeeID");
                    alAttributesToLoad.Add("givenName");
                    alAttributesToLoad.Add("mail");
                    alAttributesToLoad.Add("manager");
                    alAttributesToLoad.Add("middleName");
                    alAttributesToLoad.Add("mobile");
                    alAttributesToLoad.Add("name");
                    alAttributesToLoad.Add("physicalDeliveryOfficeName");
                    alAttributesToLoad.Add("sn");
                    alAttributesToLoad.Add("telephoneNumber");
                    alAttributesToLoad.Add("adspath");

                    using (DirectoryEntry dirEntry = CreateDirectoryEntry(strLDAPPath, strUserId, strPwd))
                    {
                        Console.WriteLine("DirectoryEntry created");

                        using (DirectorySearcher dirSearcher = CreateSearcher(dirEntry, false))
                        {
                            Console.WriteLine("DirectorySearcher created");

                            using (SearchResultCollection colResults = GetObjectsWithFilter(dirSearcher, strFilter, alAttributesToLoad, nMaxPageSize, nPageTimeLimitInSecs))
                            {
                                foreach (SearchResult result in colResults)
                                {
                                    col = result.Properties["adspath"];
                                    if ((col != null) && (col.Count > 0))
                                    {
                                        strItemPath = col[0] as string;
                                    }

                                    nCount++;
                                   
                                    //I do a bunch of stuff here to create objects in my system using the info from each LDAP object
                                    System.Threading.Thread.Sleep(6000);
                                }
                            }

                            Console.WriteLine("");
                            Console.WriteLine("Done with import");
                        }
                    }
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex.Message);
                }

                Console.WriteLine(nCount.ToString() + " items imported.");

                Console.WriteLine("Press any key to continue.");
                Console.ReadLine();
            }

            private DirectoryEntry CreateDirectoryEntry(string strImportStart, string strUserName, string strPassword)
            {
                object o = null;
                DirectoryEntry dirEntry = new DirectoryEntry(strImportStart);
               
                dirEntry.AuthenticationType = (System.DirectoryServices.AuthenticationTypes.Secure |
                        System.DirectoryServices.AuthenticationTypes.Sealing |
                        System.DirectoryServices.AuthenticationTypes.Signing);
                dirEntry.Username = strUserName;
                dirEntry.Password = strPassword;

                o = dirEntry.NativeObject; //This forces the bind and will throw an exception if bind is not valid.

                return dirEntry;
            }

            private DirectorySearcher CreateSearcher(DirectoryEntry dirEntry, bool bFollowReferrals)
            {
                DirectorySearcher dirSearcher = new DirectorySearcher(dirEntry);
                if (bFollowReferrals)
                    dirSearcher.ReferralChasing = ReferralChasingOption.All;
                else
                    dirSearcher.ReferralChasing = ReferralChasingOption.None;

                return dirSearcher;
            }

            private SearchResultCollection GetObjectsWithFilter(DirectorySearcher dirSearcher, string strSearchFilter, ArrayList alAttributesToLoad, int nMaxPageSize, int nPageTimeLimitInSecs)
            {
                SearchResultCollection colResults = null;

                dirSearcher.Filter = strSearchFilter;
                dirSearcher.SearchScope = SearchScope.Subtree;

                if (alAttributesToLoad != null)
                {
                    foreach (string strProperty in alAttributesToLoad)
                    {
                        dirSearcher.PropertiesToLoad.Add(strProperty);
                    }
                }

                dirSearcher.PageSize = nMaxPageSize;
                dirSearcher.ServerPageTimeLimit = TimeSpan.FromSeconds(nPageTimeLimitInSecs);

                Console.WriteLine("");
                Console.WriteLine("ABOUT TO IMPORT");
                Console.WriteLine("Asynchronous = " + dirSearcher.Asynchronous.ToString());
                Console.WriteLine("Cache Results = " + dirSearcher.CacheResults.ToString());
                Console.WriteLine("Client Timeout = " + dirSearcher.ClientTimeout);
                Console.WriteLine("Filter = " + dirSearcher.Filter);
                Console.WriteLine("Page Size = " + dirSearcher.PageSize);
                Console.WriteLine("Referral Chasing = " + dirSearcher.ReferralChasing.ToString());
                Console.WriteLine("Search Scope = " + dirSearcher.SearchScope.ToString());
                Console.WriteLine("Server Page Time Limit = " + dirSearcher.ServerPageTimeLimit);
                Console.WriteLine("Server Time Limit = " + dirSearcher.ServerTimeLimit);
                Console.WriteLine("Size Limit = " + dirSearcher.SizeLimit);
                Console.WriteLine("Tombstone = " + dirSearcher.Tombstone);
               
                colResults = dirSearcher.FindAll();
               
                return colResults;
            }
        }
    }

  •  06-25-2008, 2:39 PM 4033 in reply to 4031

    Re: MaxQueryDuration timeout of 2 minutes and searching

    I think the part where you are pausing 6 seconds before retrieving the next result could be a problem.  If I send you a page with 1000 results and you pause 6 seconds on each one, that is 6000 seconds before it completes a single page which is well over the timeout.  What happens if you just comment that out as well as the ServerPageTimeLimit (which shouldn't matter, btw)?  It should complete, right?

    To fix this - you would need to pull the results down as quick as possible, get what you need in the loop (store it in a collection if you need to), and then do your other processing on that data.  It is just a simple reordering of your logic.

    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
  •  06-25-2008, 3:07 PM 4041 in reply to 4033

    Re: MaxQueryDuration timeout of 2 minutes and searching

    I only put the sleep in because I was trying to get code that would time out on my customer's system.  I do not really do a "sleep."  Instead, I take the data out of the SearchResult and put it into my software system. But, that does take a few milliseconds  for each SearchResult.

    I thought you previously said that the timeout only applies to a page of data.  But, now you say that I need to pull down results as quick as possible.  Is this because the MaxQueryDuration really does apply to the whole query?

    I thought if I looped through the SearchResults, it would "behind the scenes" pull down data a page at a time.  The timeout would apply to that page, but once it came back I could use that page of data as long as I needed.  Then, when I got to a search result it hadn't pulled yet, it would do a query to pull another page of data.



  •  06-25-2008, 3:16 PM 4042 in reply to 4041

    Re: MaxQueryDuration timeout of 2 minutes and searching

    I realize that you aren't actually using Thread.Sleep(), but my point was that if you are emulating the amount of time that it takes to do your work, you will timeout.

    The reason is that enumeration (the foreach loop) is what actually initiates the network call.  It won't call for the next page until you have enumerated the entire loop of the first page.  It doesn't call each page, buffer them, and wait for you (unless you access a property like Count, which requires the enumeration first).  This is a good thing as it allows you to abandon the search before all the results are retrieved.

    There was a mistake in the book at one point where we weren't explicit about this, but I am pretty sure we fixed it in the 2nd edition to explain this behavior.

    So, ultimately, it boils down to this:  (PageSize * ms of Work inside loop) > 120 seconds == timeout


    Ryan Dunn
    Extemporaneous Mumblings
    The .NET Developer's Guide to Directory Services Programming
View as RSS news feed in XML