Skip to main content

How to deal with large document libraries in SharePoint Online

If you are reading this post, then you most likely committed the most famous sin in SharePoint – exceeded the item threshold in a document library. Don’t worry, we all make mistakes in life (well except for my mother-in-law, she claims that she never does). So I am here to forgive you 🙂  and explain what this issue is all about, how to mitigate it, and how to deal with large document libraries in SharePoint in general.

What is this item limit threshold on a document library?

It all started with SharePoint 2010. To maintain the performance of a document library (and the servers behind the scenes), Microsoft introduced the 5,000 view item limit on a document library. What that meant was that while you could put more than 5,000 items in a single library, you could not surface up (display) more than 5,000 items on a view. I actually wrote a separate post on the 5,000 view item limit issue back in 2015.

large document libraries in SharePoint

What happens if I put more than 5,000 items in a document library

Classic SharePoint/Classic Document Library

In the past, with older versions of SharePoint, the classic library pretty much became unusable. Yes, you could put 20,000 documents in a library, but if you tried to do anything in that library (search, set unique permissions, create views, sort, filter, group metadata, etc.) – you would get an error message every little step of the way.

Modern Document Library

A breakthrough occurred in 2017 with the introduction of a Modern Document Library. All of a sudden, it became more “tolerant” towards the larger volumes of data. Over the years, Microsoft made additional improvements to the modern document library, like auto-indexing libraries that contain less than 20,000 documents.

With the modern document library, you can now upload 20K, 30K, 50K documents. However, the limits are still there, and chances are – you will encounter the error messages described below.

What happens if I put a large number of documents in a document library

The structure of a library matters

So while 5,000 is no longer a hard limit, there is still a limit to how many docs a document library can handle. There is no exact number I can give you where you will get to see the errors. It depends on how you structure your library, whether or not files are organized in folders or metadata, and whether or not you sync your library to the computer.

OneDrive sync makes the issues even worse

Another factor to consider is that often users Sync the libraries to their computer via OneDrive Sync. This exacerbates the issue as OneDrive sync has limitations of its own. For example, when you have lots of files and folders in a single folder, you will get an error message below

Microsoft also recommends that users do not sync more than 300,000 files across all SharePoint document libraries/OneDrive. So the larger the library, the sooner your users will encounter the issues.

large document libraries in SharePoint

Microsoft recommendation taken from their SharePoint limits article

Common Error messages with large document library threshold

Below I would like to explain the most common error messages you will see (and respective operations that cause them).

  1. The user tries to move/reorganize folders with a large number of documentslarge document libraries in SharePoint
  2. The user tries to create unique permissions for the folder or a library that contains more than 100,000 items. The second image (captured from Mircosoft SharePoint limits post) summarizes this hard limitlarge document libraries in SharePoint
  3. The user tries to rename a folder with a large library or folder that contains a large number of itemslarge document libraries in SharePoint
  4. The user tries to move/reorganize files and folders within a document library (takes a lot of time)
  5. The user tries to rename a folder (takes a lot of time)
  6. The user tries to set unique security on a folder (takes a lot of time)
  7. The user tries to bulk edit metadata for multiple fileslarge document libraries in SharePoint
  8. The user tries to sort and group on some metadata columns (which reminds me, I need to buy myself a bicycle)large document libraries in SharePoint
  9. The user tries to create a custom view and hide folderslarge document libraries in SharePoint
  10. The user tries to create a Calculated Columnlarge document libraries in SharePoint
  11. The user gets an error message about too many files in a folder while syncing using OneDrive sync client
  12. The user gets an error message about sync issues in OneDrive
  13. The user gets a warning/error message about sync delays due to high service activity
  14. The user unable to access SharePoint sites and gets Throttle message when trying to access SharePoint (while also syncing a large document library)large document libraries in SharePoint
  15. The user’s OneDrive sync wheel is always spinning, freezes sometimes

How to properly mitigate large document library issue

If you read my old post on 5,000 view item limit, as well as posts written by others on the same topic, there are some techniques available to improve the usability of a large library. Things like indexing columns, creating custom views, limiting the number of items in a view. That said, don’t waste your time on any of them. They will only provide marginal improvements (if any), and your users will still encounter most of the issues/error messages above.

There is only one fool-proof way to avoid and mitigate the issue:

Do not put more than 20-30K files and folders into a single library. Create many sites and libraries.

Just like with driving, if you do not want to be caught speeding, well, then maybe you should not be speeding! Same thing with SharePoint document libraries. Don’t do it! Instead, concentrate on the following:

  1. Think in terms of sites and not folders.
  2. Create many sites and many libraries, spread the content among them
  3. Don’t use SharePoint like Dropbox or file share
  4. Spend some time building proper Information Architecture
  5. Train your users on dos and don’ts of SharePoint

You may also like

Teams Wiki vs. OneNote

September 10th, 2020

You may also like

Top 5 limitations of SharePoint Online

August 20th, 2020

You may also like

Why you should not remove default features in SharePoint

August 6th, 2020

Need SharePoint Help?

Hourly consulting, training and configuration services are available

Learn More