| Real-time Speech recognition | Speech synthesis | Voice activity detection |
|---|---|---|
| ✔️ | ✔️ | ✔️ |
| Architecture | Android | iOS | Windows | macOS | linux |
|---|---|---|---|---|---|
| x64 | ✔️ | ✔️ | ✔️ | ✔️ | |
| x86 | ✔️ | ✔️ | |||
| arm64 | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ |
| arm32 | ✔️ | ✔️ | |||
| riscv64 | ✔️ |
| 1. C++ | 2. C | 3. Python | 4. JavaScript |
|---|---|---|---|
| ✔️ | ✔️ | ✔️ | ✔️ |
| 5. Go | 6. C# | 7. Kotlin | 8. Swift |
|---|---|---|---|
| ✔️ | ✔️ | ✔️ | ✔️ |
It also supports WebAssembly.
This repository supports running the following functions locally
- Streaming speech-to-text (i.e., real-time speech recognition)
- Text to speech (e.g., vits models from piper)
- VAD (e.g., silero-vad)
on the following platforms and operating systems:
- x86,
x86_64, 32-bit ARM, 64-bit ARM (arm64, aarch64), RISC-V (riscv64) - Linux, macOS, Windows, openKylin
- Android, WearOS
- iOS
- NodeJS
- WebAssembly
- Raspberry Pi
- RV1126
- LicheePi4A
- VisionFive 2
- 旭日X3派
- etc
with the following APIs
- C++, C, Python, Go,
C# - Kotlin
- JavaScript
- Swift
We support all platforms that ncnn supports.
Everything can be compiled from source with static link. The generated executable depends only on system libraries.
HINT: It does not depend on PyTorch or any other inference frameworks other than ncnn.
Please see the documentation https://k2-fsa.github.io/sherpa/ncnn/index.html for installation and usages, e.g.,
- How to build an Android app
- How to download and use pre-trained models
We provide a few YouTube videos for demonstration about real-time speech recognition
with sherpa-ncnn using a microphone:
-
Multilingual (Chinese + English) with endpointing Python demo : https://www.bilibili.com/video/BV1eK411y788/
-
Android demos
-
Multilingual (Chinese + English) Android demo 1: https://www.bilibili.com/video/BV1Ge411A7XS
-
Multilingual (Chinese + English) Android demo 2: https://www.bilibili.com/video/BV1eK411y788/
-
Chinese (with background noise)Android demo : https://www.bilibili.com/video/BV1GR4y167fx -
ChineseAndroid demo : https://www.bilibili.com/video/BV1744y1Z76H -
Chinese poem with background musicAndroid demo : https://www.bilibili.com/video/BV1vR4y1k7eo
| Description | URL |
|---|---|
| Streaming speech recognition | Address |
https://github.com/k2-fsa/sherpa-ncnn/releases/tag/models
- Documentation: https://k2-fsa.github.io/sherpa/ncnn/
- Bilibili 演示视频: https://search.bilibili.com/all?keyword=%E6%96%B0%E4%B8%80%E4%BB%A3Kaldi
Please see https://k2-fsa.github.io/sherpa/social-groups.html for 新一代 Kaldi 微信交流群 and QQ 交流群.