Camera-based health data recording

Measure health markers like blood pressure, BMI, heart rate (HR), heart rate variability (HRV), and more in just 30 seconds with a face scan using the technology of our partner Shen.ai.

Allow users to record clinically validated vital signs like blood pressure, heart rate, and heart rate variability by scanning their face within your application, just as if they are FaceTiming with a loved one.

You can use the face-scanning technology within your application by adding the ThryveShenAI module to the SDK.

If you would like to understand more on how to get access to ShenAI technology via Thryve, please reach out to our business development team.

Integrate ShenAI with your application

To integrate the camera-based health data with your iOS app, you need to integrate the ThryveShenAI module in addition to the ThryveCore and ThryveCommons modules. Follow either the cocoapods or framework files instructions in the iOS section.

Since it is Camera based you need to ask the user permission to access the Camera. Add the Privacy - Camera Usage Description key entry in your info.plist file with the value as the description of why you need to access the user's Camera. See the screenshot below for refrence.

Configure the `ThryveShenAI` module

When initializing the Thryve SDK ensure to set your ShenAI API key to shenAIConfig and an optional ThryveShenAIEventListener in the ThryveShenAIConfig .

import ThryveCore
import ThryveCommons
import ThryveObservability
import ThryveShenAI

let shenAIConfig = ThryveShenAIConfig(
    apiKey: "SHEN_AI_API_KEY",
    eventsListener: self // Optional: make sure your class or struct conform to `ThryveShenAIEventListener` protocol and implements `onEvent(event: ThryveShenAIEvent)` function
)

let thryveSDKConfig = ThryveSDKConfig(
    authId: "AUTH_ID",
    authSecret: "AUTH_SECRET",
    endUserAlias: "YOUR_UNIQUE_USER_IDENTIFIER",
    endUserId: nil, 
    locale: "de",
    configs: [shenAIConfig],
    observability: ObservabilityConfig(tracingEnabled: true,              crashReportingEnabled: true)
)
        
let thryveSDK = await ThryveSDK.getOrCreate(thryveSDKConfig)

// Optionally, when ThryveSDK is initialized from a non-async context (AppDelegate, legacy code, etc.), it is recommended to use the getOrCreate(...) callback to know when the SDK is ready.

ThryveSDK.getOrCreate(thryveSDKConfig) { initResult in
    if initResult.successful {
        Logger.i("ThryveSDK ready, you can start calling API methods")
    } else if let error = initResult.errors?.first {
        Logger.e("ThryveSDK init failed: \(error.errorMessage ?? "Unknown error")")
    }
}

import com.thryve.sdk
import com.thryve.sdk.commons
import com.thryve.sdk.shenAI
   
val shenAIConfig = ThryveShenAIConfig(
    apiKey = "SHEN_AI_API_KEY",
    eventsListener = object : ThryveShenAIEventListener {
        override fun onEvent(event: ThryveShenAIEvent) {
            when (event) {
                ThryveShenAIEvent.MEASUREMENT_FINISHED -> { /* Shen AI measurement has finished and is about to be uploaded */}
                ThryveShenAIEvent.MEASUREMENT_FAILED -> { /* Shen AI measurement has failed - handle failure */}
                ThryveShenAIEvent.DATA_UPLOAD_FINISHED -> { /* Shen AI has finished uploading data */}
                ThryveShenAIEvent.DATA_UPLOAD_FAILED -> { /* Shen AI has failed uploading data */}
                ThryveShenAIEvent.USER_SUMMARY_FINISHED -> { /* The Shen AI flow has been completed */ }
            }
        }
    }
 )
    
val thryveSDKConfig = ThryveSDKConfig(
    authId = "ASSIGNED_AUTH_ID",
    authSecret = "ASSIGNED_AUTH_SECRET",
    endUserAlias = "XXXXXXXXXXX",
    endUserId = null, 
    locale = "de",
    bleConfig = bleConfig
)
                 
val thryveSDK = ThryveSDK.getOrCreate(thryveSDKConfig, context)

/**Android SDK 5.0.5 introduced an optional callback to communicate the status 
  of internal processes of getOrCreate. ThryveSDK instance can now be created 
  with a callback as shown in the sample code below.  
**/
val thryveSDK = ThryveSDK.getOrCreate(thryveSDKConfig, context) { thryveResponse ->
       if(thryveResponse.successful){
           //ThyveSDK initialization processes completed successfully
       } else {
           // ThryveSDK initalization process failed. process the ThryveErrors for the specific reason.
           thryveResponse.errors.map { thryveError -> Logger.e(TAG){" getOrCreate ThryveError in onCreate function $thryveError"} }
       }
  }

Parameter

Description

Mandatory

apiKey

The apiKey as provided by ShenAI. Please reach out to our support if you're interested in obtaining an api key for ShenAI.

yes

eventsListener

An object of ThryveShenAIEventListener that will be receiving the following events from the ShenAI module.

MEASUREMENT_FINISHED
MEASUREMENT_FAILED
DATA_UPLOAD_FINISHED
DATA_UPLOAD_FAILED
USER_FLOW_FINISHED

Trigger camera-based health data measurement

To record and upload all available camera-based health data use the measure function

import ThryveCore
import ThryveShenAI
import ThryveCommons

...
func measure() {
    ThryveSDK.get().measure { response in
        if response.successful, let success = response.data, success {
            Logger.d { "Camera data Measure: Measurement and data upload finished successfully.")
        } else {
            Logger.d { "Camera data Measure failed: \(String(describing: response.errors?.first?.errorMessage ?? "Unknown error"))")")
        }
    }
}
...

import com.thryve.sdk.ThryveSDK
import com.thryve.sdk.shenAI

...
fun measure(activity: ComponentActivity){
    ThryveSDK.get()?.measure(activity) {
        runOnUiThread {
            Logger.d([email protected]) { "Camera data Measure successful = ${it.successful} data = ${it.data}" }
            it.errors.map { error ->
                Logger.d([email protected]) { "Camera data Measure error $error" }
            }
        }
    }
}
...

import { ThryveSDK } from '@thryve/react-native-sdk';

//measure Camera based Health data
export async function measure() {
  const sdk = new ThryveSDK().getOrCreate(thryvSDKConfig);
  try {
    const response = await sdk.measure();
    if (response?.data) {
      //measure was successfull
    } else {
      //measure failed. process all errors.
      response.errors.forEach((error) => {
        console.log(`Camera Health Data measure error ${error}`);
      });
    }
  } catch (err) {
    console.error('❌ Unexpected error', err);
  }finally {
      sdk.finish();
  }
}

import 'package:thryve_sdk/thryve_sdk.dart';

final class ShenAIPageBloc {
  final ThryveSDK _thryveSDK;
  final Source _dataSource;

  ShenAIPageBloc(this._thryveSDK) : _dataSource = Source.shenAI;

  Future<ThryveResponse<void>> measure() => _thryveSDK.measure(_dataSource.id);

   Future<void> measureCameraHealthData() async {
    final ThryveResponse<void> response = await measure();

    if (!response.isSuccessful) {
      final Iterable<ThryveError> errors = response.errors.whereType<ThryveError>();

      for (final ThryveError error in errors) {
        //process each error here
      }

      throw StateError('Unable to measure health data using ShenAI.');
    }
  }
}

Calibration

The ShenAI technology features a calibration option to enhance the accuracy of blood pressure measurements. The feature is triggered using the calibrate method of the Thryve SDK. During calibration, users are guided through a process where they provide readings from an external blood pressure monitor. This aids in refining the precision of camera-based measurements. Note that calibration is optional.

import ThryveCore
import ThryveShenAI
import ThryveCommons

...
func calibrate() {
    ThryveSDK.get().calibrate { response in
        if response.successful, let success = response.data, success {
            Logger.d { "Camera data Calibrate: Measurement and data upload finished successfully.")
        } else {
            Logger.d { "SCamera data Calibrate failed: \(String(describing: response.errors?.first?.errorMessage ?? "Unknown error"))")")
        }
    }
}
...

import com.thryve.sdk.ThryveSDK
import com.thryve.sdk.shenAI

...
fun calibrate(activity: ComponentActivity){
    ThryveSDK.get()?.calibrate(activity) {
        runOnUiThread {
            Logger.d([email protected]) { "ShenAI Calibrate successful = ${it.successful} data = ${it.data}" }
            it.errors.map { error ->
                Logger.d([email protected]) { "ShenAI Calibrate error $error" }
            }
        }
    }
}
...

PreviousReal-time heart rate data NextError handling

Last updated 20 days ago

hashtagIntegrate ShenAI with your application

hashtagConfigure the ThryveShenAI module

hashtagTrigger camera-based health data measurement

hashtagCalibration

Integrate ShenAI with your application

Configure the `ThryveShenAI` module

Trigger camera-based health data measurement

Calibration