Conversion formulas between RGB and YCbCr involve offsets. So even if it is all zeros in one domain you will not get all zeros in the other domain. If you directly use the formula to convert an all zero YCbCr domain matrix to RGB you would get positive green values and negative red and blue values. These negative values are usually thresholded to 0 which is what you are seeing.
The wikipedia page http://en.wikipedia.org/wiki/YCbCr and MATLAB Color Space Conversion block reference page at http://www.mathworks.com/help/vision/ref/colorspaceconversion.html shows the formulas used.