Nico, you were right. But I had MANY MANY more issues that were creeping up on me.
Issue #1: I wasn't calculating my normals properly. I was using the per vertex average but wasn't even aware that there was such a technique as weighted average. This was a 2000% improvement on all my lighting.
Issue #2: Tangent and Bitangent calculation were not being done correctly either. I might still improve on that area to see if I can also do a weighted average of them.
Issue #3: I wasn't doing my lighting calculations correctly and after being on Wikipedia for about 2 days, I finally did it right, and actually understand it now with complete clarity.
Issue #4: I just wanted to hurry up and do it without understanding 100% what I was doing(never making that mistake AGAIN).