This will stop working when you will ever delete a project (for example a C# project) that has been built at least once from the Team Foundation Project. The next Get operation on the build server will try to remove the project folder from the build dir. It does so quite cleverly by removing all the files it knows about in a bottom up manner. Unfortunately there are some files created by the previous build, that Source Control does know nothing about. The last build left something like "obj\debug" behind. When the SC Get operation tries to remove the directory that contains "obj\debug" the operation fails and so the build breaks. You have to manually delete the directory from the build dir to make the build work again.
I would not bother with incrememtal builds anyway if you are looking for a fast build. There is much more time wasted by the build service by transmitting the current build status to the tfs server than by recompiling the sources.