The normal “EAS Build” service is currently something you have to pay for. They do intend to have a “free tier” in future. But yes, you can also build locally on your own machine for free.
For building iOS apps locally you will need a Mac.
For building Android apps, you should in theory be able to do it on any OS that can run a current version of Android Studio.