convo birngs together silero and rasa to create continuous speech conversationalist experience like Alexa or Google dot.

• silero STT and TTS models provide the quality comparable to Google's STT (and sometimes even better) but they are not Google. See silero performance benchmarks

• rasa is an enterprise-grade chatbot built on python and Transformer based language models providing state-of-the-art framework comparable or better than top cloud based chatbot frameworks

convo can run easily on a local cpu based machine, thus convo provides high response times at no cloud service costs.

Typical STT and TTS inference time on a local machine for one sentence is less than 0.5 seconds each and rasa bot response time is around 1-2 seconds. This can be improved even further by fine tuning and using dedicated machines.

There is some time duration required for the user to speak and for convo to play response audio. This does not count as processing time of convo or any other voicebot.

convo advantages:

• High performance as the framework can run locally on a cpu;
• No cloud charges so this can be implemented for masses;
• highly customizable using rasa custom action server to add any desired functionality;
• Can support multiple languages as supported by silero models

convo does not use any hotword detection mechanism however it can stop speaking by speaker requesting with keywords like stop / quit / exit.

There are 2 base softwares / frameworks those need to be installed for setting up convo

• rasa
• silero

1. Create a python virtual environment named "rasa" with suitable python version mentioned in rasa installation here. Current rasa version 3.x requires python 3.7 or 3.8. enable rasa virtual environment before following below installation steps.

2. Install rasa using below commands

 pip install --upgrade pip
 
 pip install rasa

3. Run "rasa init" on the terminal. please follow on screen instructions to complete creating rasa chatbot instance.

 rasa init

4. Once you have rasa chatbot instance installed you can check if it is working properly by running rasa shell that lets you talk to your assistant on the command line. Note - Change the directory to your rasa instance and run below command.

 rasa shell

This will run rasa server and let you chat with it on terminal. Please enter "/stop" to stop rasa server.

5. We would be calling this rasa chatbot using rest api call. When we want to communicate with rasa chatbot, we will need to start rasa using

 rasa run --enable-api

1. Create a python virtual environment named "silero" with latest python version

2. There are quite a few dependencies for running silero. we will go through them in following steps

3. Install pytorch using instructions on https://pytorch.org/get-started/locally/ - if you are on windows & cpu only, this command may look like below. Use appropriate command line for your machine configuration.

pip install torch torchvision torchaudio

4. Additionally we need following python packages

pip install PySoundFile SpeechRecognition omegaconf

speechRecognition is a wrapper liberary that allows performing speech recognition using multiple ASR services including google cloud speech etc. We will only be using this liberarty to capture and record audio since it provides detecting voice activity and ending mic recording when user stops speaking.

5. we also need pyaudio installed. On windows 10-11 you may encounter error installing pyaudio. Please use following commands to install pyaudio in that case.

pip install pipwin 
    
pipwin install pyaudio 

6. convo uses imports from silero those are already included in this repo. please check an ensure that silero model and utils are at the right place

With that we are done with the installation steps. Now try running convo.py in the terminal using silero virtual environment and you should be able to speak with your computer 👍

python convo.py

when you run convo for the first time, it will download silero models to cache. Download Progress will be displayed in the output terminal. In subsequent runs it will use locally cached models which will be fast.

if you are not able to speak with your computer then try checking below points

1. Please check if your mic and speaker are enabled. on windows you may also need to check permissions etc.
2. Please check if all the mentioned liberaries are installed properly and you are running both silero and rasa in their own virtual environments
3. Please check if are running rasa server from inside of the rasa bot directory using "rasa run --enable-api" and it said rasa server is up and running
4. Standard laptop mic doesn't often have a great quality that may impact speech recognition quality. Try raising mic input volume level or try using better quality mic.
5. If you have had some compatibility errors while installing on the virtual environments, you may want to delete and recreate them
6. In future there might be a change in the avaliable liberaries or compatibility, please do check for those kind of issues.

This repo builds on top of 2 great softwares

• https://github.com/RasaHQ

• https://github.com/snakers4/silero-models

This repo presents the base working implementation of convo. This can be further enhanced in many ways. Some of the enhancements are mentioned below

• Add more functionality to rasa like chitchat, faq and custom api calls
• Add more languages and speakers
• Perform in momery processing of audio

• If you face any issues or have suggesions, please mention those in issues section
• If you liked this repo and/or you were able to use it in your work, please consider starring and mentioning this repo in your ciatations