Got this TFS build error, “Delete Sources Directory … The directory is not empty” suddenly. It was strange because there were no changes being checked in for the past few days.
Researched and found the article  mentioned that there are some processes such as file explorer or command prompt might have handles on file system. Followed the hint did solve the build error.
I looked back and asked so why didn’t it happen earlier? I believed it was because there were a few build agents, and it took a few rounds to reuse the same folder.