The reason you are getting an error is because the type of $rootBuildPath when you get it from the xml file is string. This results in the equivalent of calling
Get-ChildItem '\\devint1\d`$\Builds\' -Exclude "Latest" | ...
which will throw the exception you are seeing. The reason it doesn't throw an error when you run
Get-ChildItem \\devint1\d`$\Builds\ -Exclude "Latest" | ...
from the command line is powershell is parsing the path as a path first before handing it to the Get-ChildItem commandlet.
In order to make your code work, you must remove the errant '`' from the path before calling Get-ChildItem.