Does GPT-3 encoder work in react native? "require module "fs (node standard library) "

How to / Third Party Tooling
1

This says on the openAi page: “If you need a programmatic interface for tokenizing text, check out our tiktoken package for Python. For JavaScript, the gpt-3-encoder package for node.js works for most GPT-3 models.” . I have installed it to test but when I try to import it I get this error in the console: “The package at “node_modules\gpt-3-encoder\Encoder.js” attempted to import the Node standard library module “fs”.
It failed because the native React runtime does not include the Node standard library.
Learn more: Use libraries - Expo Documentation” Is there a way to make it work or is it just not possible to use it in react native?

or is there any other way to tokenize a text for the openAi api locally in my expo application?

2

The problem is these lines:

You would need to replace those. React Native does not implement Node’s fs library. Perhaps you could use Expo FileSystem or else include the files as Assets

You could then use patch-package to patch gpt-3-encoder.

1 Like
3

I am trying to access the file using expo-asset but I get an error, what could be happening? my code is: `
import * as FileSystem from ‘expo-file-system’;
import { Asset } from ‘expo-asset’;

async function loadAppJs() {
const asset = Asset.fromModule(require(‘…/assets/niveles.json’));
await asset.downloadAsync();
const JsonContent = await FileSystem.readAsStringAsync(asset.localUri);
console.log(‘Contenido de niveles.json:’, JsonContent);
}
loadAppJs();`
levels.json is inside the assets folder in the main directory, also in app.json add: “assets”: [
“./assets/levels.json”
], “assetBundlePatterns”: [
“assets/levels.json”
],
*in the console it shows me the following: "Possible Unhandled Promise Rejection (id: 2):
Error: Module “abra,abre,abri,acne,acre,acta,acto,adan,afan,afaz,afea,afin,afro,afta,agil,agro,agua,ague,aguao, aire,ajea,ajis,ajos” is missing from the asset registry
(it is showing all content of niveles.json after ‘error: Module’)

4

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.