1 Overview
2 Server Side
- 2.1 Creating Token
- 2.2 Creating User
- 2.3 Request For Score
3 App Side
- 3.1 iOS
  - 3.1.1 Recording Wav File
  - 3.1.2 Uploading Wav File
  - 3.1.3 Request For Score
- 3.2 Android
  - 3.2.1 Recording Wav File
  - 3.2.2 Uploading Wav File
  - 3.2.3 Request For Score

Overview

High level sequence diagram is shown below for How you can integrate your AppServer and App with SondePlatform

Server Side

Creating Token

Using java-sdk, access token can be generated as below:

Replace <clientId> and <clientSecret> in below snippets by the actual values that you must have received

SondeCredentialsService cred = SondeHealthClientProvider.getClientCredentialsAuthProvider(<clientId>, <clientSecret>);

Make a list of all the required scopes. Example below

List<Scopes> scopeList = Arrays.asList(Scopes.STORAGE_WRITE, Scopes.SCORES_WRITE, Scopes.MEASURES_READ, Scopes.MEASURES_LIST);

Now, generate access token using the SondeCredentialsService

AccessToken token = null;
try{
	token = cred.generateAccessToken(scopeList);
	String accessToken = token.getAccessToken(); //This line will get the accessToken with the requested scopes
	}
catch(SondeServiceException | SDKClientException | SDKUnauthorizedException ex){
	//Exception handling code
}

Creating User

Instantiate the SondeCredentialsService using your clientId and clientSecret

SondeCredentialsService cred = SondeHealthClientProvider.getClientCredentialsAuthProvider(<clientId>, <clientSecret>);

Create a subject client

UserClient userClient = SondeHealthClientProvider.getSubjectClient(cred);

Build the UserCreationRequest using the user attributes.

UserCreationRequest request = new UserCreationRequest.UserBuilder(Gender.MALE,"1991").withLanguage("ENGLISH").build(); // language is optional

Create user using sujectclient created in step 2

UserCreationResponse response;
try{
  userClient = SondeHealthClientProvider.getUserClient(cred);
  response = userClient.createUser(request);
}
catch(SondeServiceException | SDKClientException | SDKUnauthorizedException ex){
	//your exceptional handling code
}

Request For Score

Instantiate the SondeCredentialsService using your clientId and clientSecret

SondeCredentialsService cred = SondeHealthClientProvider.getClientCredentialsAuthProvider(<clientId>, <clientSecret>);

Upload the file using Storage Client

try{
  StorageClient client = SondeHealthClientProvider.getStorageClient(cred);
  FileUploadRequest fileUploadRequest = new FileUploadRequest(Country.INDIA, <user Identifier>, "C:\\test.wav", FileType.WAV);
  User identifier is the one that was generated when we created subject For example - "3cf7e4d31"
  FileUploadResponse response = client.uploadFile(fileUploadRequest);
}
catch(SondeServiceException | SDKClientException | SDKUnauthorizedException ex){
	//your exceptional handling code
}

Create a Scoring client and get Score for a measure

try{
  ScoringClient scoringClient = SondeHealthClientProvider.getScoringClient(cred);
  ScoringRequest request = new ScoringRequest.Builder(response.getFilePath(), "emotional-resilience").withUserIdentifier("3cf7e4d31").build();
  ScoringResponse scoringResponse = scoringClient.getScore(request);
}
catch(SondeServiceException | SDKClientException | SDKUnauthorizedException ex){
	//your exceptional handling code
}

App Side

iOS

Recording Wav File

Wave file should be recorded in sampling rate of 44100 Hz

Adding the audio recording permission description into plist file

<key>NSMicrophoneUsageDescription</key>
  <string>Your message to show in permission dialog</string>

Ask recording permission from the user

func askForAudioRecordingPermission(){
        do{
            try recordingSession.setCategory(.record)
            recordingSession.requestRecordPermission({allowed in
                if !allowed{
                    self.showPermissionError()
                }
            })
        }catch{}
    }

To record the audio file follow the code snippet given below

func startRecording() {
    let recordingSession = AVAudioSession.sharedInstance()
    var audioRecorder: AVAudioRecorder!
        let settings: [String:Any] = [
            AVFormatIDKey: kAudioFormatLinearPCM,
            AVLinearPCMBitDepthKey:16,
            AVSampleRateKey: 44100.0,
            AVNumberOfChannelsKey: 1 as NSNumber,
            AVEncoderAudioQualityKey: AVAudioQuality.high.rawValue,
            AVAudioFileTypeKey: kAudioFileWAVEType
        ]
       
        do{
            audioRecorder = try AVAudioRecorder(url: audioFilePath!, settings: settings)
            audioRecorder.prepareToRecord()
            audioRecorder.record()
            
        }catch{}
    }

Stop Audio Recording

audioRecorder.stop()

Uploading Wav File

– Parameters

– countryCode:

– subjectIdentifier:

To upload the audio file you need to first get the pre-signed URL where you need to upload the file.
Upload the file from the path where your audio file is created.

  func getSignedURL(countryCode: String, fileType: String, subjectIdentifier: String, completion:@escaping (_ storageResponse:[String:AnyObject])->Void, errorCompletion:@escaping (_ error:String)->Void){
        guard let url = URL(string: baseURL + "storage/files") else {
            return
        }
        if accessToken.isEmpty{
            return
        }
        
        let body:[String: String] = ["countryCode": countryCode, "fileType": fileType, "subjectIdentifier": subjectIdentifier]
        guard let bodyData = try? JSONSerialization.data(withJSONObject: body, options: []) else{
            return
        }
        
        
        let session = URLSession.shared
        var request = URLRequest(url: url)
        request.httpMethod = "POST"
        request.setValue(accessToken, forHTTPHeaderField: "Authorization")
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        request.httpBody = bodyData

        session.dataTask(with: request,completionHandler: {(data, response, error) in
            if error != nil{
                errorCompletion(error!.localizedDescription)
            }else{
                if let data = data, let storageResponse = try? JSONSerialization.jsonObject(with: data, options: []) as? [String: AnyObject]{
                   completion(storageResponse)
                }else{
                    errorCompletion("Invalid Response")
                }
            }
        }).resume()
    
        
    }

Upload the audio file

– Parameters:

– signedURL: You get this value from the response of getSignedURL.

– audioFileURL: URL path where you recorded your audio file.

 func uploadFile(signedURL:String, audioFileURL:URL, completion:@escaping ()->Void, errorCompletion:@escaping (_ error:Error)->Void){
        guard let  uploadURL = URL(string: signedURL) else{
            return
        }
        if accessToken.isEmpty{
            return
        }
        
        let session = URLSession(configuration: .default)
        var request = URLRequest(url: uploadURL)
        request.httpMethod = "PUT"
        
        session.uploadTask(with: request, fromFile: audioFileURL, completionHandler: {(data, response, error) in
            if (response as? HTTPURLResponse)?.statusCode == 200{
                completion()
            }else{
                errorCompletion(error!)
            }
        }).resume()
    }

Request For Score

– Parameters:

– fileLocation: You can get the file location from the response of getSignedURL.

– measureName: The value of a measure for which you want to calculate the score.

  func getScore(fileLocation:String, measureName:String, completion:@escaping (_ scoreResponse: [String:AnyObject])->Void, errorCompletion:@escaping (_ error:String)->Void){
        guard let url = URL(string: baseURL + "inference/scores") else {
            return
        }
        
        let body: [String: AnyObject] = ["fileLocation": fileLocation as AnyObject, "measureName" : measureName as AnyObject]
        guard let bodyData = try? JSONSerialization.data(withJSONObject: body, options: []) else{
            return
        }
        if accessToken.isEmpty{
            return
        }
        
        let session = URLSession(configuration: .default)
        var request = URLRequest(url: url)
        request.httpMethod = "POST"
        request.httpBody = bodyData
        request.setValue(accessToken, forHTTPHeaderField: "Authorization")
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        
        session.dataTask(with: request,completionHandler: {(data, response, error) in
            if error != nil{
               errorCompletion(error!.localizedDescription)
            }else{
                if let data = data, let responseJSON = try? JSONSerialization.jsonObject(with: data, options: []) as? [String: AnyObject]{
                    completion(responseJSON)
                }else{
                    
                    errorCompletion("Invalid Response")
                }
            }
        }).resume()
        
    }

Android

Recording Wav File

Wave file should be recorded in sampling rate of 44100 Hz

You can record a audio file in following steps :

i). Request for microphone permission :

To record an audio, you must request for microphone permission on android 6 and above.

Add the permission in manifest

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

Below code snippet you can refer to request for microphone permission.

  private void requestAudioPermissions() {
        if (ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {

            //When permission is not granted by user, show them message why this permission is needed.
            if (ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.RECORD_AUDIO)) {
                Toast.makeText(this, "Please grant permissions to record audio", Toast.LENGTH_LONG).show();
                ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.RECORD_AUDIO}, PERMISSIONS_RECORD_AUDIO);

            } else {
                // Show user dialog to grant permission to record audio
                ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.RECORD_AUDIO}, PERMISSIONS_RECORD_AUDIO);
            }
        }
        //If permission is granted, then go ahead for recording audio
        else if (ContextCompat.checkSelfPermission(this,
                Manifest.permission.RECORD_AUDIO)
                == PackageManager.PERMISSION_GRANTED) {
            // start Recording audio;
        }
    }

ii). Start recording of audio :

Below snippet you can refer to start recording of audio and writing it to a file.

The expected file is .WAV file, you should convert your PCM file to .WAV file by using method WaveUtils.pcmToWave() (You can find implementation of this method in GitHub sample project here: https://github.com/sondehealth-samples/score/blob/master/client/android/app/src/main/java/com/sonde/sample/utils/WaveUtils.java ) This method adds required headers in the file.

// starts recording a file at internal memory
    private void startRecording() {
        mAudioRecord = new AudioRecord(AUDIO_SOURCE, SAMPLE_RATE_HZ, AUDIO_CHANNEL_CONFIG, AUDIO_FORMAT, AUDIO_BUFFER_SIZE_BYTES);
        mAudioRecord.startRecording();
        mFilePath = getFilesDir().getAbsolutePath() + "/" + System.currentTimeMillis() + ".wav";
        recording = true;
        new Thread(new Runnable() {
            @Override
            public void run() {
                writeAudioDataToFile(mFilePath);
            }
        }).start();
    }
    
    private void writeAudioDataToFile(String filename) {
        String audioFilename = filename + ".pcm";
        OutputStream outputStream = null;
        try {
            outputStream = new FileOutputStream(audioFilename);
            int bufferSizeBytes = AUDIO_BUFFER_SIZE_BYTES;
            short[] audioBuffer = new short[bufferSizeBytes / 2]; // assumes 16-bit encoding
            byte[] outputBuffer = new byte[bufferSizeBytes];
            int totalBytesRead = 0;
            while (recording) {
                int numShortsRead = mAudioRecord.read(audioBuffer, 0, audioBuffer.length);

                for (int i = 0; i < numShortsRead; i++) {
                    outputBuffer[i * 2] = (byte) (audioBuffer[i] & 0x00FF);
                    outputBuffer[i * 2 + 1] = (byte) (audioBuffer[i] >> 8);
                    audioBuffer[i] = 0;
                }

                int numBytesRead = numShortsRead * 2;
                totalBytesRead += numBytesRead;
                outputStream.write(outputBuffer, 0, numBytesRead);
            }

            OutputStream waveOutputStream = new FileOutputStream(mFilePath);
            InputStream dataInputStream = new FileInputStream(audioFilename);
            short numChannels = 1;
            short sampleSizeBytes =  2;
            //convert pcm to wav file
            WaveUtils.pcmToWave(waveOutputStream, dataInputStream, totalBytesRead, numChannels, SAMPLE_RATE_HZ, sampleSizeBytes);
        } catch (Exception e) {
            Log.e(TAG, "Error : " + e);
        }finally {
            try {
                if (outputStream != null) {
                    outputStream.close();
                }
            } catch (IOException e) {
            Log.e(TAG, "Error : " + e);
            }
            try {
                if (mAudioRecord != null) {
                    mAudioRecord.stop();
                    mAudioRecord.release();
                    mAudioRecord = null;
                }
            } catch (IllegalStateException e) {
            Log.e(TAG, "Error : " + e);
            }
            // delete PCM file
            File file = new File(audioFilename);
            file.delete();
        }
    }

Uploading Wav File

Get pre-signed url from server to upload file, give details like location, userIdentifier, and fileType

private void getS3SignedUrl( {
        BackendApi backendApi = RetrofitClientInstance.getRetrofitInstance().create(BackendApi.class);
        String countryCode = "IN"; //Use your country code
        Call<S3PathResponse> call = backendApi.getS3FilePath(accessToken, new S3FilePathRequest("wav", countryCode, userIdentifier));
        call.enqueue(new Callback<S3PathResponse>() {
            @Override
            public void onResponse(Call<S3PathResponse> call, Response<S3PathResponse> response) {
                S3PathResponse s3PathResponse = response.body();
                if (s3PathResponse != null) {
                String s3FilePath = s3PathResponse.getSignedURL();
                    //use the received signed url to upload wav file
                } else {
                    Log.e(TAG, "error: code " + response.code());
                }
            }

            @Override
            public void onFailure(Call<S3PathResponse> call, Throwable t) {
                Log.e(TAG, "error: " + t);
            }
        });
    }

Once you got the S3 signed url, next step is to upload wav file to S3 bucket.

You can refer below code snippet to upload file to S3 bucket

private void uploadFileToS3(final S3PathResponse s3PathResponse, final File filePath) {
        BackendApi backendApi = RetrofitClientInstance.getRetrofitInstance().create(BackendApi.class);
        MediaType MEDIA_TYPE_OCTET_STREAM = MediaType.parse("application/octet-stream");
        String uploadUrl = s3PathResponse.getSignedURL();
        Call<ResponseBody> call = backendApi.uploadFileToS3(uploadUrl, RequestBody.create(MEDIA_TYPE_OCTET_STREAM, filePath));
        call.enqueue(new Callback<ResponseBody>() {
            @Override
            public void onResponse(Call<ResponseBody> call, Response<ResponseBody> response) {
                if (response.isSuccessful()) {
                    Log.i(TAG, " : File Uploaded successfully " + filePath.getName());
                    // request for measure score
                } else {
                    Log.e(TAG, " : Failed to upload file " + filePath.getName() + " Error code : " + response.code());
                }
            }

            @Override
            public void onFailure(Call<ResponseBody> call, Throwable t) {
                Log.e(TAG, " : Failed to upload file " + filePath.getName() + " Error: " + t);
            }
        });
    }

Request For Score

After successfully uploading the wav file, the last step requires you to request for measure score.

You can refer below code snippet to request for measure score.

 private void requestForMeasureScore(String fileLocation, final String measureName, String userIdentifier) {
        Log.i(TAG, " : fileLocation : " + fileLocation);
        BackendApi backendApi = RetrofitClientInstance.getRetrofitInstance().create(BackendApi.class);
        Call<InferenceScoreResponse> call = backendApi.getInferenceScore(accessToken, new InferenceScoreRequest(fileLocation, measureName, userIdentifier));
        call.enqueue(new Callback<InferenceScoreResponse>() {
            @Override
            public void onResponse(Call<InferenceScoreResponse> call, Response<InferenceScoreResponse> response) {
                InferenceScoreResponse scoreResponse = response.body();
                if (scoreResponse != null) {
                    //show calculated score for measure
                } else {
                    Log.e(TAG, " : Failed to get score , Error: " + response.code());
                    Toast.makeText(MainActivity.this, "Could not calculate the score, please try again", Toast.LENGTH_LONG).show();
                }
            }

            @Override
            public void onFailure(Call<InferenceScoreResponse> call, Throwable t) {
                Log.e(TAG, " : Failed to get score , Error: " + t);
                Toast.makeText(MainActivity.this, "Could not calculate the score, please try again", Toast.LENGTH_LONG).show();
            }
        });
    }

For more information, please contact Sonde at support@sondehealth.com.

Functionality code snippets

Overview

Server Side

Creating Token

Creating User

Request For Score

App Side

iOS

Recording Wav File

Uploading Wav File

Request For Score

Android

Recording Wav File

Uploading Wav File

Request For Score

Related content