Hi. As @adamjnav says here if you do not use the Facebook (or other) SDK in your app to display ads then you should tell Google that your app does not display ads.
The only way currently to remove the Facebook SDK is to eject, but just having the Facebook SDK as part of your app should not be a problem if you don’t use it.