It sounds like you may be running into this bug- though it’s hard to tell for certain. If you have thousands of files, I’m about certain that’s the problem.
We’re on that one, so I’d suggest subscribing to the bug report. As soon as there’s a patch it will be posted there.
Packet Tide owns and develops ExpressionEngine. © Packet Tide, All Rights Reserved.