Efficient way to retrieve all list items

Question

I am retrieving all items from list containing around 4000 items. But it seems to take longer time to fetch all items which is ~15 to ~22 seconds. Is there any best way to fetch all items from list in negligible time?

Following is the code i am using to fetch all items:

 using (SPSite spSite = new SPSite(site))
 {
     using (SPWeb web = spSite.OpenWeb())
     {
         list = web.Lists["ListName"];
         SPQuery query1 = new SPQuery();
         string query = "<View>";
         query += "<ViewFields>";
         query += "<FieldRef Name='ID' />";
         query += "<FieldRef Name='Title' />";                    
         query += "</ViewFields>";
         query += "<Query>";
         query += "<Where>";
         query += "<Eq>";
         query += "<FieldRef Name='ColName'></FieldRef>";
         query += "<Value Type='Boolean'>1</Value>";
         query += "</Eq>";
         query += "</Where>";
         query += "</Query>";

         query += "</View>";
         query1.Query = query;
         SPListItemCollection listItems = list.GetItems(query1);
    }
}

Answer 1

Make sure that the column you include in the where clause is defined as an indexed column in SharePoint. You can check by going to the list you're querying and into List -> Settings -> Indexed Columns. It's not applicable in your CAML query, but if you have more than one 'where', ensuring the first where hits an indexed column can increase efficiency.

Refer to Microsoft's guidance on designing large lists here.

Answer 2

Using CAML query is the best option for retrieving data.

As list grows if the result set has more than 5000 items, then performance will degrade. To handle this situation as SCMC suggested you should add indexes to fields used in the WHERE clause.

If you still observe performance issues, the I would suggest using Pagination. So get items in set to X items. You can configure X using CAML RowLimit attribute.