Where to add FCM server key if hosting our own server instead of pushing to expo sever?

We have an expo project where it is hosted on our own server using expo export to bundle the files and deploy it to S3. We are implementing the Push Notification on Android and iOS and have it setup already. The notification is working on iOS but not on Android. In the documentation, the FCM server key should be uploaded on expo server (which I did) to make the server communicate with the app and make the push notification work. I am wondering if the uploaded server key is also getting called and referenced even if the app is hosted on S3 not on Expo? I was thinking it was the cause of the push notification not working on Android. I did some experimenting, publishing the app on expo server and building an android apk and the push notification is working otherwise if I do the exporting on our own server and build an apk, the push notification is not working anymore.