Overview

The following use case is a guide for your app development. The below use case will provide detailed information for recording the audio sample and generating the score for Mental Fitness health conditions.

Notes:

Notes:
- Please check Sonde EDGE SDK setup at Android EDGE SDK

As a developer, I want to measure voice characteristics of user’s mental-fitness health condition

Perform the below steps on the app side to generate the score:

STEP 1: Display the prompt and duration to the user
STEP 2: Record 30 seconds audio file
STEP 3: Generate the inference score for Mental Fitness
STEP 4: Display the score to the user.

STEP 1: Display the prompt and duration to the user

We recommend displaying the prompt along with the duration of the recording to the user before starting the actual recording. This will help the user to get ready to speak on the recording screen. Every health condition has its own prompt(s) and timer length of recording.

You can get the required prompt to be shown to the user on the instruction screen and timer length by referring Vocal Biomarkers - Health Checks

Refer to the screenshot below for your reference.

STEP 2: Record 30 seconds audio file

Before starting the audio recording, add the following check to make sure that the RECORD_AUDIO permission is enabled in your app:

Kotlin

if (ContextCompat.checkSelfPermission(this,
  android.Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
    val permissions = arrayOf(android.Manifest.permission.RECORD_AUDIO)
    ActivityCompat.requestPermissions(this, permissions, REQUEST_CODE_RECORD_AUDIO)
}

Java

if (ContextCompat.checkSelfPermission(getActivity(),
  Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
    String[] permissions = {Manifest.permission.RECORD_AUDIO};
    ActivityCompat.requestPermissions(getActivity(), permissions, REQUEST_CODE_RECORD_AUDIO);
}

Also, verify that the app user grants record audio permission—the recording functionality will not function without the permission. To do this, override the onRequestPermissionsResult method in your app and check the permission result.

Kotlin

override fun onRequestPermissionsResult(
  requestCode: Int,
  permissions: Array<String>,
  grantResults: IntArray
) {
  super.onRequestPermissionsResult(requestCode, permissions, grantResults)

  if (requestCode == REQUEST_CODE_RECORD_AUDIO && grantResults.isNotEmpty()
      && grantResults[0] != PackageManager.PERMISSION_GRANTED) {
    throw RuntimeException("Record Audio permission is required to start recording.")
  }
}

Java

Initialise the SondeAudioRecorder and call the beginRecording() function to start recording.

beginRecording() function requires the parameters : timeInMillis, and timerRecordingListener .

timeInMillis : Length of the audio recording in milliseconds e.g. 30000 (for 30 seconds audio file)

timerRecordingListener : This listener provides the callbacks for the recorder initialized, recording started, for any errors, and when recording finishes.

Kotlin

Once done with recording for a given duration, recording will stop and you will get the callback at onRecordingFinish() function with the filePath of the recorded audio sample.

STEP 3: Generate the inference score for Mental Fitness

Once you are done with recording the audio file, you can generate the score for Mental Fitness. To generate the score, follow the below steps:

a. Create an instance of Inference

createInference() function accepts two parameters application context and metadata

Kotlin

(Must call SondeEdgeSdk.init before creating inference engine)

Gender is required parameter can be passed as Gender.MALE or Gender.FEMALE.

Birth year is an optional parameter, you must send a positive four digit integer in it.

partnerId is an optional parameter of type string.

b. Call the inferScore() function

inferScore() function requires audioFilePath : the generated audio file path from the above step, healthCheckType : health condition for which you want to generate the score (here MENTAL_FITNESS), and InferenceCallback : callback when the score is generated and callback when any exception occurs

Kotlin

(Must create InferenceEngine before calling infer score)

Typecast the score to VFFinalScore to get the other voice features scores.

Kotlin

Voice Sufficiency Check

We have introduce voice sufficiency checks in our SDK, enhancing the audio quality for Mental Fitness Score assessments, ensuring only audio files containing speech and meeting the SNR threshold are getting scored.
Developers needs to integrate error responses ( Low Speech; High Noise; Low Speech and High Noise) mentioned in section “object : InferenceCallback “.

STEP 4: Display the score to the user

The value of the score generated in step 3 varies between 0-100. Once you get the score, display it to the user. You should always show the interpretation message on the score screen to the user. For some applications, scores may be uploaded and not shown to the users. However, the scores can be uploaded for an individual user or in bulk for further analysis through API functions calls.

Refer Vocal Biomarkers - Health Checks for the interpretation message.

Below is the screenshot for your reference.

Sonde Health Platform Documentation

Mental Fitness Android EDGE Use Case

Overview

As a developer, I want to measure voice characteristics of user’s mental-fitness health condition

Voice Sufficiency Check