Build a Voice Assistant Mobile App with React Native and ChatGPT Integration
“This is a part of my training programme in 42Interactive.com, and 42Interactive.com do these kind of services”
Overview:
In this tutorial, we’ll delve into the intricacies of leveraging React Native and seamlessly integrating ChatGPT to create an intelligent voice-driven application. From creating project to crafting interactive voice interactions, you’ll gain insights into the seamless marriage of React Native’s capabilities with the powerful language models of ChatGPT.
What is a voice assistant?
A voice assistant is a digital assistant that uses speech recognition, natural language processing, and artificial intelligence (AI) to understand and respond to spoken commands or queries. These assistants are designed to perform various tasks and provide information based on user input.
Voice assistants can be found in various devices, including smartphones, smart speakers, smart TVs, and other internet-connected devices. Some well-known examples of voice assistants include: Amazon Alexa, Apple Siri, Google Assistant, Microsoft Cortana, Samsung Bixby.
Voice assistants have become increasingly popular due to advancements in natural language processing and the widespread adoption of smart devices.
What is ChatGPT?
ChatGPT is a language model developed by OpenAI, based on the GPT (Generative Pre-trained Transformer) architecture. The “Chat” in ChatGPT signifies its focus on generating human-like text for conversational purposes. It is designed to understand and generate coherent responses in natural language, making it suitable for chat-based applications.
ChatGPT was created by OpenAI an artificial intelligence (AI) research organization that aims to advance digital intelligence in a safe and beneficial manner. It was founded in December 2015 by Elon Musk, Sam Altman, Greg Brockman, Ilya Sutskever, John Schulman, and Wojciech Zaremba, among others. OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all of humanity.
The application will look like below:
Prerequisites:
- Node.js and npm: Ensure that Node.js and npm are installed on your machine.
- Text Editor: Choose a text editor of your preference; Visual Studio Code is recommended.
- OpenAI API Key: Obtain your OpenAI API key by signing up at https://openai.com.
Create a New React Native Project
Begin by creating a new React Native project using the following commands
npx react-native init ReactNativeVoiceAssistantAfter finish, next run the project whit this commands:
cd RectNativeVoiceAssistant
npx react-native run-androidIt will showing default screen like below:
Open Project with Visual Studio Code
Open the project with Visual Studio Code. Here we can see some files and folders that were automatically generated from the previous step.
Installing Dependencies
Dependencies are external libraries or packages that enhance the functionality of your application, providing pre-built solutions for common tasks.
// for interaction with Rest API
npm install axios
// speech to text
npm install @react-native-voice/voice
// react native navigation
npm install @react-navigation/native
npm install react-native-screens
npm install react-native-safe-area-context
npm install @react-navigation/native-stack
// text to speech
npm install --save react-native-tts
// chat component
npm install react-native-gifted-chat
// vector icons
npm install --save react-native-vector-icons
npm install @types/react-native-vector-icons
// environment variable
npm install -D react-native-dotenv
Here, I will not provide details about each library mentioned above. You can refer to their respective official sites for documentation.
Implementing Voice Assistant
Creating Additional Folder
Let’s create some folders: src/screens, src/navigations, src/services, src/styles, src/store, src/utils, and src/components to better organize our code. In the screens folder, we will place all code related to the screen, the navigations folder for code related to navigation, services for code related to accessing the rest API for OpenAI, styles for stylesheets, and store for Redux state management.
Creating Service for Accessing the ChatGPT REST API
Creating this service is a fundamental step in enabling seamless communication with the ChatGPT API, allowing your tutorial mobile app to leverage the power of ChatGPT for an interactive and dynamic conversational experience.
First, let’s create a file named .env. Inside it, there should be a constant for accessing OpenAI. Yes, it should be without single or double quotes. Please remember not to publish your API key.
API_KEY=YourApiKeyHere
API_URL=https://api.openai.com/v1/chat/completionsCreate the file src/services/OpenAiService.js. The code will read a variable from the .env file and use Axios to call the API, returning the response from OpenAI. If an error occurs, it will return an error message.
import {API_URL, API_KEY} from "@env"
import axios from "axios";
export const getAnswerFromGpt = async (prompt) => {
try {
const client = axios.create(
{
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${API_KEY}`,
},
}
);
const response = await client.post(API_URL, {
"model": "gpt-3.5-turbo",
"messages": [
{
role: "assistant",
content: prompt
}
]
});
const answer = response.data?.choices[0]?.message?.content;
return Promise.resolve({ success: true, data: answer });
} catch (error) {
return Promise.resolve({ success: false, msg: error.message });
}
}Creating a Welcome Screen
Creating a welcome screen in a mobile app involves designing a screen that serves as the initial interface for users when they launch your app. Welcome screens typically provide a warm introduction, branding, and often include features like onboarding or sign-in options. Here is the welcome screen I will create.
Lets create file on folder src/screens with filename WelcomeScreen.tsx use this code:
// Import necessary components and libraries
import { NavigationProp, ParamListBase, useNavigation } from '@react-navigation/native';
import React from 'react';
import { View, Text, TouchableOpacity } from 'react-native';
import { SafeAreaView } from 'react-native-safe-area-context';
import MaterialCommunityIcons from 'react-native-vector-icons/MaterialCommunityIcons';
import { styles } from '../styles/WelcomeStyle';
const WelcomeScreen = () => {
const navigation = useNavigation<NavigationProp<ParamListBase>>();
const handleNext = () => {
navigation.navigate('ChatWithVoiceScreen');
};
return (
<SafeAreaView style={styles.container}>
<View style={styles.container}>
<View>
<Text style={styles.title}>React Native Voice Assistant</Text>
<Text style={styles.subtitle}>With Voice Command powered by OpenAI</Text>
</View>
<MaterialCommunityIcons name="account-tie-voice" size={200}
style={{ marginBottom: 10, marginRight: 10, color: '#10a37f' }}
/>
<TouchableOpacity style={styles.button} onPress={handleNext}>
<Text style={styles.buttonText}>Start Chat</Text>
</TouchableOpacity>
</View>
</SafeAreaView>
);
};
export default WelcomeScreen;In the code snippet above, you can observe our utilization of Navigation, Stylesheet, and MaterialCommunity icons as components of the Vector Icon.
import { StyleSheet } from 'react-native';
// Styles
const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'space-between',
alignItems: 'center',
padding: 38,
},
title: {
fontSize: 24,
fontWeight: 'bold',
textAlign: 'center',
color: '#10a37f'
},
subtitle: {
fontSize: 12,
color: '#666',
textAlign: 'center',
},
button: {
backgroundColor: '#10a37f',
paddingVertical: 5,
paddingHorizontal: 20,
borderRadius: 20,
},
buttonText: {
color: '#FFF',
fontSize: 18,
},
});
export { styles }Creating a Voice Assistant Screen
Now, we are creating a screen for our voice assistant application, located in the src/screens folder with the filename ChatWithVoiceScreen.tsx. This screen will manage voice recognition, converting spoken words to text, facilitating chat typing, activating voice settings, and, of course, presenting response results from ChatGPT. Additionally, it will include text-to-speech functionality to transform written text into spoken words. Please look at red dot, it all features of our screen.
This screen is organized into seven main sections, each serving a specific function. Starting from the top:
- User Chat: Input from either typing or voice command.
- ChatGPT Responses: Displayed with corresponding avatars.
- Scroll-down Icon: Allows scrolling to the latest response.
- Typing Input: Field for entering text.
- Send Button: Initiates the sending of messages.
- Voice Command Button: Press to start recording voice commands, and press again to stop recording.
Here the code:
// Import necessary components and libraries
import Voice, { } from '@react-native-voice/voice';
import React, { useCallback, useEffect, useState } from 'react';
import { View, Text, Platform, PermissionsAndroid, Rationale, TouchableHighlight, SafeAreaView, TouchableOpacityBase, TouchableOpacity } from 'react-native';
import { getAnswerFromGpt } from '../services/OpenAiService';
import {
Bubble,
GiftedChat,
IMessage,
Send,
} from 'react-native-gifted-chat'
import MaterialCommunityIcons from 'react-native-vector-icons/MaterialCommunityIcons';
import { TypingAnimation } from 'react-native-typing-animation';
import { styles } from '../styles/ChatWithVoiceStyle';
import { useSelector } from 'react-redux';
import Tts from 'react-native-tts';
import { initTts, readText } from '../utils';
const initialMessages: IMessage[] =
[
{
_id: 1,
text: 'Hi there! How can I assist you today?',
createdAt: new Date(),
system: true,
user: { _id: 1 }
},
];
const ChatWithVoiceScreen = () => {
const [recording, setRecording] = useState(false);
const [errorMsg, setErrorMsg] = useState('');
const [messages, setMessages] = useState(initialMessages);
const [answering, setAnswering] = useState(false);
const [result, setResult] = useState('');
const [speaking, setSpeaking] = useState(false);
const [ttsStatus, setTtsStatus] = useState('');
const voice = useSelector((state: any) => state.voice.value);
const onSend = useCallback((messages: IMessage[] = []) => {
const { text, } = messages[0];
setMessages((previousMessages) => {
return GiftedChat.append(previousMessages, messages, Platform.OS !== 'web');
})
processTranscription(text);
}, []);
useEffect(() => {
Voice.onSpeechStart = (e) => {
setErrorMsg('');
setRecording(true);
};
Voice.onSpeechEnd = (e) => {
setRecording(false);
};
Voice.onSpeechError = (e: any) => {
const errMsg: string = e.error?.message;
if (errMsg.includes('No match')) {
setErrorMsg("You are not speaking!");
} else {
setErrorMsg(errMsg);
}
setRecording(false);
}
Voice.onSpeechResults = (e: any) => {
const prompt = e.value[0];
if (!prompt) {
return;
}
setResult(prompt);
};
setMessages(initialMessages);
return () => {
Voice.destroy().then(Voice.removeAllListeners);
Tts.stop();
};
}, []);
const stopRecording = async () => {
console.log("== stopRecording ");
try {
await Voice.stop();
setRecording(false);
console.log("== stopRecording: ", result);
const newMsg = {
_id: Math.round(Math.random() * 1000000),
text: result,
createdAt: new Date(),
user: {
_id: 2,
name: 'User',
}
};
const newMessage = [newMsg]
setMessages((previousMessages) => {
return GiftedChat.append(previousMessages, newMessage, Platform.OS !== 'web');
})
if (result) {
processTranscription(result);
}
} catch (error: any) {
console.log("== eror when stop: ", error);
setErrorMsg(error.message)
}
}
const startRecording = async () => {
console.log("== startRecording ");
setRecording(true);
Tts.stop();
try {
await Voice.start('en-US');
} catch (e) {
console.error(e);
}
};
const stopSpeaking = () => {
Tts.stop();
setSpeaking(false);
}
const readTheAnswer = (message: string) => {
setSpeaking(true);
readText(message);
}
const processTranscription = async (prompt: string) => {
if (prompt.trim().length > 0) {
setAnswering(true);
getAnswerFromGpt(prompt.trim()).then((res: any) => {
if (res.success) {
setAnswering(false);
const newMsg = {
_id: Math.round(Math.random() * 1000000),
text: res.data,
createdAt: new Date(),
user: {
_id: 1,
name: 'Assistant',
}
};
const newMessage = [newMsg]
setMessages((previousMessages) => {
return GiftedChat.append(previousMessages, newMessage, Platform.OS !== 'web');
})
if (voice) {
readTheAnswer(res.data);
}
} else {
setAnswering(false);
setErrorMsg(res.msg);
}
})
}
}
const renderSend = (props: any) => {
return (
<>
<Send {...props}>
<MaterialCommunityIcons name="send"
style={styles.buttonSend}
/>
</Send>
<TouchableOpacity
style={styles.buttonMicStyle}
onPress={recording ? stopRecording : startRecording}>
{recording ?
<MaterialCommunityIcons name='stop' style={styles.buttonRecordingOff} /> :
<MaterialCommunityIcons name='microphone' style={styles.buttonRecordingOn} />}
</TouchableOpacity>
</>
);
};
const scrollToBottomComponent = () => {
return <MaterialCommunityIcons name="arrow-down-circle-outline" size={38} color="#10a37f" />;
};
return (
<SafeAreaView style={styles.container}>
<View style={styles.container}>
<GiftedChat
messages={messages}
showAvatarForEveryMessage={true}
onSend={messages => onSend(messages)}
user={{
_id: 2,
name: 'User',
avatar: ''
}}
alwaysShowSend
renderSend={renderSend}
scrollToBottom
scrollToBottomComponent={scrollToBottomComponent}
/>
</View>
</SafeAreaView>
);
};
export default ChatWithVoiceScreen;Creating a Navigation
Lets create file AppNavigation.tsx on folder src/navigations
import React from 'react';
import { NavigationContainer } from '@react-navigation/native';
import { createNativeStackNavigator } from '@react-navigation/native-stack';
import WelcomeScreen from '../screens/WelcomeScreen';
import ChatWithVoiceScreen from '../screens/ChatWithVoiceScreen';
const Stack = createNativeStackNavigator();
const AppNavigation = () => {
return (
<NavigationContainer>
<Stack.Navigator initialRouteName="WelcomeScreen">
<Stack.Screen name="WelcomeScreen" component={WelcomeScreen} options={{ headerShown: false }}/>
<Stack.Screen name="ChatWithVoiceScreen" component={ChatWithVoiceScreen}
options={{
title: 'Voice Assistant',
headerStyle: {
backgroundColor: '#d8d8d8',
},
headerTintColor: 'black',
headerTitleStyle: {
fontWeight: 'bold',
},
}}
/>
</Stack.Navigator>
</NavigationContainer>
);
};
export default AppNavigation;Next we integrating the Navigation to our application, modify the App.tsx file to include the AppNavigation component:
import { SafeAreaProvider } from "react-native-safe-area-context";
import { createNativeStackNavigator } from "@react-navigation/native-stack";
import AppNavigation from "./src/navigations/AppNavigation";
const Stack = createNativeStackNavigator();
export default function App() {
return (
<SafeAreaProvider>
<AppNavigation/>
</SafeAreaProvider>
);
}Run Your React Native App
To set up your environment, run, and build a React Native project, you can follow the instructions outlined in the React Native Get Started guide.
run android:
npx react-native run-androidrun iOS:
npx react-native run-iosThe Demo Video https://www.youtube.com/watch?v=1UBG3eRddS8
Conclusion:
Congratulations on constructing a sophisticated voice assistant utilizing React Native and the ChatGPT API. Users can effortlessly engage with the application through voice commands, and the system adeptly transcribes and processes spoken words using OpenAI’s state-of-the-art language models.
Consider this tutorial as a springboard for the development of voice-enabled applications, seamlessly integrating them with the advanced natural language processing capabilities offered by OpenAI. Feel empowered to tailor and expand the application according to your unique specifications, exploring additional features, refining the user interface, and fortifying error handling for a refined and responsive user experience. The potential for customization is vast, and we encourage you to unleash your creativity to enhance the functionality and overall appeal of your voice assistant application.
“This is a part of my training programme in 42 Interactive.com, and 42Interactive.com do these kind of services”
