Condivisione della tecnologia

[UNI-APP] Modulo dattiloscritto per dettatura di frasi Alibaba NLS

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

I codici demo forniti da Alibaba sono tutti JavaScript, quindi puoi crearne uno tuo. Come riferimento, ho scritto un modulo dattiloscritto per la dettatura delle frasi Alibaba Nls. Il modulo API combinato di VUE3

startClient: avvia la dettatura. Tieni presente che il passaggio successivo deve essere quello di abilitare il riconoscimento e la trasmissione dei dati il ​​prima possibile, altrimenti verrà chiuso dopo 6 secondi.

startRecognition: avvia la transazione di riconoscimento, passa la richiamata di riconoscimento, puoi stampare i caratteri o visualizzarli sullo schermo

sendSound: invia dati PCM binari (formato 16 MHz 16 bit)

stopRecognition: termina la transazione di riconoscimento

  1. /**
  2. * 阿里语音,一句话识别模块for ccframe
  3. *
  4. * 无心跳设计,非长连接推送,因此在需要使用的时候才进行连接
  5. *
  6. * @Jim 2024/07/08
  7. */
  8. import * as utils from '@/utils/index'
  9. import { nextTick } from 'vue'
  10. // import Global from '@/utils/constants'
  11. const NLS_SERVER_URL = 'wss://nls-gateway.aliyuncs.com/ws/v1'
  12. const NLS_MODE = 'SpeechRecognizer' // 一句话识别
  13. const WEBSOCKET_MAX_RETRY = 3
  14. const RECONNECT_INTERVAL = 3000
  15. interface INlsConfig {
  16. url?: string
  17. appkey: string // 应用的key
  18. token: string // 从服务器获得,要缓存
  19. }
  20. let client: (UniNamespace.SocketTask & { readyState?: WsState }) | undefined
  21. const clientId = utils.uuid(utils.UUIDFormat.StandardCompact)
  22. let taskId: string = ''
  23. let config: INlsConfig
  24. let reconnectAttempts = 0
  25. let taskStarted = false
  26. enum WsState {
  27. CONNECTING,
  28. OPEN,
  29. CLOSING,
  30. CLOSED
  31. }
  32. /**
  33. *
  34. * @param action
  35. * @returns 请求json
  36. */
  37. const buildMsg: (action: string, payload: Record<string, any>) => string = (
  38. action,
  39. payload = {}
  40. ) => {
  41. if (taskId.length === 0) {
  42. taskId = utils.uuid(utils.UUIDFormat.StandardCompact)
  43. }
  44. const msg = {
  45. header: {
  46. message_id: utils.uuid(utils.UUIDFormat.StandardCompact),
  47. task_id: taskId,
  48. namespace: NLS_MODE,
  49. name: action,
  50. appkey: config.appkey
  51. },
  52. payload,
  53. context: {
  54. sdk: {
  55. name: 'nls-wx-sdk',
  56. version: '0.0.1',
  57. language: 'wxjs'
  58. }
  59. }
  60. }
  61. return JSON.stringify(msg, null, 0)
  62. }
  63. /**
  64. * 开启连接,开启后立即要传,否则会被关闭.
  65. * @param config
  66. * @param callback
  67. */
  68. export const startClient = (
  69. conf?: INlsConfig,
  70. startCallback?: () => void,
  71. recognizedCallback?: (text: string) => void
  72. ) => {
  73. if (client && client.readyState !== WsState.CLOSED) {
  74. // 关闭原连接
  75. client.close({})
  76. }
  77. client = uni.connectSocket({
  78. url: conf.url ?? NLS_SERVER_URL,
  79. tcpNoDelay: true,
  80. header: {
  81. 'X-NLS-Token': conf?.token ?? config.token
  82. },
  83. success: (res) => {
  84. if (!config) config = conf
  85. console.log(`connected to ${NLS_SERVER_URL} success`)
  86. },
  87. fail: (res) => {
  88. console.log(`connect to ${NLS_SERVER_URL} failed:${res.errMsg}`)
  89. }
  90. })
  91. client.readyState = WsState.CONNECTING
  92. client.onMessage((res) => {
  93. if (typeof res.data === 'string') {
  94. const msgObj = JSON.parse(res.data)
  95. switch (msgObj?.header?.name) {
  96. case 'RecognitionStarted': {
  97. console.log('started')
  98. break
  99. }
  100. case 'RecognitionResultChanged': {
  101. if (recognizedCallback) {
  102. const text = msgObj?.payload?.result
  103. if (text) {
  104. recognizedCallback(text)
  105. }
  106. }
  107. console.log('changed')
  108. break
  109. }
  110. case 'RecognitionCompleted': {
  111. const text = msgObj?.payload?.result
  112. if (text) {
  113. recognizedCallback(text)
  114. }
  115. taskStarted = false // 结束识别
  116. break
  117. }
  118. case 'TaskFailed': {
  119. taskStarted = false // 结束识别
  120. break
  121. }
  122. }
  123. }
  124. console.log('recv:' + res.data)
  125. })
  126. client.onOpen(() => {
  127. reconnectAttempts = 0
  128. client.readyState = WsState.OPEN
  129. if (startCallback) nextTick(startCallback)
  130. })
  131. client.onError((error) => {
  132. console.error('WebSocket error:', error)
  133. if (reconnectAttempts < WEBSOCKET_MAX_RETRY) {
  134. setTimeout(() => startClient(), RECONNECT_INTERVAL)
  135. } else {
  136. console.error('Max reconnect attempts reached')
  137. }
  138. })
  139. client.onClose(() => {
  140. client.readyState = WsState.CLOSED
  141. console.log('connection closed')
  142. })
  143. }
  144. export const startRecognition = () => {
  145. if (client && client.readyState === WsState.OPEN)
  146. client.send({
  147. data: buildMsg('StartRecognition', {
  148. format: 'opus',
  149. sample_rate: 16000,
  150. enable_intermediate_result: true,
  151. enable_punctuation_prediction: true,
  152. enable_inverse_text_normalization: true
  153. }),
  154. success: (res) => {
  155. taskStarted = true
  156. }
  157. })
  158. }
  159. export const stopRecognition = () => {
  160. if (client && client.readyState === WsState.OPEN)
  161. client.send({
  162. data: buildMsg('StopRecognition', {
  163. format: 'opus',
  164. sample_rate: 16000,
  165. enable_intermediate_result: true,
  166. enable_punctuation_prediction: true,
  167. enable_inverse_text_normalization: true
  168. }),
  169. complete: () => {
  170. taskStarted = false // 不管是否成功,都不发送音频了
  171. }
  172. })
  173. }
  174. export const sendSound = (msgBytes: ArrayBuffer) => {
  175. if (client && client.readyState === WsState.OPEN && taskStarted)
  176. client.send({
  177. data: msgBytes,
  178. success: (res) => {
  179. console.log('send ' + msgBytes.byteLength + ' success')
  180. }
  181. })
  182. }

Per lo strumento uuid di util, vedere il mio articolo precedenteItaliano: Italiano: Italiano: https://mp.csdn.net/mp_blog/creation/editor/140267684icona-default.png?t=N7T8Italiano: Italiano: Italiano: https://mp.csdn.net/mp_blog/creation/editor/140267684