Project Brief

According to the ASER education report 2018, 50% of students in class 5 are unable to read class 2 level text and speak English. Only 26% of the students can afford to go to private tuition and the rest cannot afford expensive tutors. In most schools, it is difficult for teachers to provide special attention to each student due to poor teacher-to-student ratio and fatigue.

The purpose of the capstone project was to develop an application to make learning fun and efficient for kids and facilitate teachers to teach better. The proposed solution is a voice-based English Language Learning app to help the primary kids to read and speak the English language effectively with the help of their mother tongue.  The use case is to create an app that will aid primary school children inaccurately reading, and pronouncing English vocabulary from the NCERT Marigold class 1 English textbook with the help of vernacular languages.

This AI-based virtual tutor app can correct the pronunciation of the child and give immediate feedback.

Approach

CRISP-DM methodology is followed to develop the project lifecycle. Python is the programming language used as the backend for developing the web app. SQL is used for creating the database. Cordova is used for developing the android based mobile app.

Natural language processing techniques are used to develop this AI (Artificial Intelligence) integrated app. Text mining techniques like tokenization, stemming, lemmatization, removal of symbols, special characters, numeric digits, etc. were used to clean the raw unstructured data. Machine learning models like Automatic Speech Recognition, Google to text converter, sequence matcher, speech synthesizer, etc. were used to build the engine of the model.

Results

The speaking, listening and reading features of the app work well with minimal flaws. With the help of a deep translator-based translation model, the words are translated to vernacular languages.  The decoder-encoder-based speech recognition models converted the text-to-speech and speech-to-text.

The app has been trained and tested for mapping the audio and text continuously to improve the accuracy level. The app performs well in a quiet atmosphere. The application uses the microphone of the mobile to capture the words spoken by the kid. Since the user interface is minimalistic, the app is user-friendly. The app can analyze and match 90% of the words with an accuracy of 100%.  The app supports five Indian languages: Kannada, Hindi, Malayalam, Tamil, and Telugu, and is under various stages of testing.

Keywords: GTTS, ASR, HMM, GMM, Text Mining, Natural Language Processing, grapheme, phoneme.