I am going to guess that what you think is called the "Compatibility library" is the Android Support package. That used to be called the Android Compatibility Library, over a year ago.
The Android Support package contains "backports" of capabilities introduced in newer versions of Android. There are other backports as well, such as ActionBarSherlock, which offers a backport of the action bar.
If you elect to use one or more backports, you are coding to the backports APIs. You are not coding to "lower API" or "most recent API", for the services provided by the backports.
My general recommendation to developers using a recent copy of the Android developer tools is to:
Set the
android:minSdkVersion
to the oldest version of Android you are willing to supportSet the build target and the
android:targetSdkVersion
to a relatively recent version of Android, one that contains features that you may wish to conditionally use on newer devicesUse the Android Support package, ActionBarSherlock, and other backports for features that they support that you want to use consistently on older and newer devices