Hey, i’m looking to build an app which recognizes a sound wave with a certain frequency and then do something if it happened- for example, sound wave with 10Hz frequency occured, send the user a notification.
I was wondering if anyone knows how i can approach this? i want to actively listen to the phone’s mic. the recognition i think i will use tensorflow (if you have other suggestions, you’re welcome to share).
thank you in advance! 
Hi
Unless I’m mistaken, Expo doesn’t have a way to analyze the audio while it’s being captured/recorded, so with Expo you’d have to record a snippet of audio, then call tensorflow (or whatever) to analyze it.
If you want to analyze the audio while it’s being captured then you’d need some native code (or an existing library that uses native code to do this). This reminds me of react-native-vision-camera’s frame processors (which allow processing frames of a video stream on the fly). Maybe there’s already something similar to this for audio.
In terms of picking out sound frequencies, you might want to look into the Fast Fourier Transform (FFT), which I believe can be used for that sort of thing.
1 Like
Hey, thank you so much for the detailed answer.
will definitely look into what you said, especially FFT.
If i do go the native route, do you know of anything that will work for android? (i’ll start with android/kotlin as i don’t have a mac for ios development)
Off hand, I don’t know. But this search turns up some useful-looking results.
Including this stack overflow answer (see also the other answers. There’s one that mentions FFT.)
Another interesting looking one: Sound recognition in Android - Stack Overflow
Others that may or may not be useful:
- A Definitive Guide for Audio Processing in Android with TensorFlow Lite Models | by Vasanthkumar Velayudham | Heartbeat
- https://nsr.cse.buffalo.edu/wp-content/uploads/2016/10/rtdroid-sound-jtres16.pdf
The above are not for React Native, so if you want to use them from a React Native app you will have to do more work. I have never done anything like that before, but Marc Rousavy’s talk (first one in this video) should help with that.
1 Like
Thank you so much friend! you gave me a lot to research and proceed from here- much appreciated!
@mortzhory I just saw this relevant-looking tweet: https://twitter.com/barthap10/status/1446396994176573465
1 Like