# Camera-based health data recording

Allow users to record clinically validated vital signs like blood pressure, heart rate, and heart rate variability by scanning their face within your application, just as if they are FaceTiming with a loved one.

<figure><img src="/files/xpn8QSyE8xllIZXJl1CD" alt="" width="188"><figcaption></figcaption></figure>

You can use the face-scanning technology within your application by adding the `ThryveShenAI` module to the SDK.

{% hint style="info" %}
If you would like to understand more on how to get access to [ShenAI](https://shen.ai/) technology via Thryve, please reach out to our business development team.
{% endhint %}

## Integrate ShenAI with your application

{% tabs %}
{% tab title="iOS" %}
To integrate the camera-based health data with your iOS app, you need to integrate the `ThryveShenAI` module in addition to the `ThryveCore` and `ThryveCommons` modules. Follow either the [cocoapods](/integrate-your-mobile-app/setup-thryve-sdk/ios.md#integration-via-cocoapods) or [framework files](/integrate-your-mobile-app/setup-thryve-sdk/ios.md#integration-via-frameworks) instructions in the [iOS section](/integrate-your-mobile-app/setup-thryve-sdk/ios.md).

{% hint style="info" %}
If `ShenaiSDK` added using `Cocoapods`, `ENABLE_USER_SCRIPT_SANDBOXING` must be set to `NO` in Xcode 15+
{% endhint %}

Since it is Camera based you need to ask the user permission to access the Camera. Add the `Privacy - Camera Usage Description` key entry in your info.plist file with the value as the description of why you need to access the user's Camera. See the screenshot below for reference.

<figure><img src="/files/yOaRB1ARnXecGJsvscST" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Android" %}
To integrate the camera-based health data with your Android app, you need to integrate the `thryve_module_shenai` module in addition to the `thryve_core_sdk` and `thryve_module_commons` modules. Integrate following either  the [Thryve Repository Dependencies ](/integrate-your-mobile-app/setup-thryve-sdk/android.md#integrate-using-thryve-repository-dependencies)or  [Thryve .aar Libraries](https://docs.thryve.health/integrate-your-mobile-app/pages/gd15Xn3eNpfQhy8TrHU5#integrate-using-thryve-.aar-libraries) instructions in the [Android section](https://open.gitbook.com/~site/site_WRtC1/mobile-apps/setup-thryve-sdk/android)
{% endtab %}

{% tab title="React Native" %}
To integrate the camera-based health data with your React Native/Expo app, you need to integrate the `@thryve/react-native-shenai-module` module in addition to the `@thryve/react-native-sdk` . Integrate the following either the [React Native CLI ](/integrate-your-mobile-app/setup-thryve-sdk/react-native.md#react-native-cli)or  [Expo](/integrate-your-mobile-app/setup-thryve-sdk/react-native.md#expo) instructions in the [React Native](/integrate-your-mobile-app/setup-thryve-sdk/react-native.md) section.
{% endtab %}

{% tab title="Flutter" %}
To integrate the camera-based health data with your Flutter app, you need to integrate the `plugin/module_shen_ai` module in addition to the `plugin/thryve_sdk` . Follow the [Flutter integration instructions ](/integrate-your-mobile-app/setup-thryve-sdk/flutter.md)in the Flutter section.
{% endtab %}
{% endtabs %}

### Configure the `ThryveShenAI` module

When initializing the Thryve SDK ensure to set your ShenAI API key to `shenAIConfig`  and an optional `ThryveShenAIEventListener` in the `ThryveShenAIConfig` .

{% tabs %}
{% tab title="iOS" %}

```swift
import ThryveCore
import ThryveCommons
import ThryveObservability
import ThryveShenAI

let shenAIConfig = ThryveShenAIConfig(
    apiKey: "SHEN_AI_API_KEY",
    eventsListener: self // Optional: make sure your class or struct conform to `ThryveShenAIEventListener` protocol and implements `onEvent(event: ThryveShenAIEvent)` function
)

let thryveSDKConfig = ThryveSDKConfig(
    authId: "AUTH_ID",
    authSecret: "AUTH_SECRET",
    endUserAlias: "YOUR_UNIQUE_USER_IDENTIFIER",
    endUserId: nil, 
    locale: "de",
    configs: [shenAIConfig],
    observability: ObservabilityConfig(tracingEnabled: true,              crashReportingEnabled: true)
)
        
let thryveSDK = await ThryveSDK.getOrCreate(thryveSDKConfig)

// Optionally, when ThryveSDK is initialized from a non-async context (AppDelegate, legacy code, etc.), it is recommended to use the getOrCreate(...) callback to know when the SDK is ready.

ThryveSDK.getOrCreate(thryveSDKConfig) { initResult in
    if initResult.successful {
        Logger.i("ThryveSDK ready, you can start calling API methods")
    } else if let error = initResult.errors?.first {
        Logger.e("ThryveSDK init failed: \(error.errorMessage ?? "Unknown error")")
    }
} 
```

{% endtab %}

{% tab title="Android" %}

```kotlin
import com.thryve.sdk
import com.thryve.sdk.commons
import com.thryve.sdk.shenAI
   
val shenAIConfig = ThryveShenAIConfig(
    apiKey = "SHEN_AI_API_KEY",
    eventsListener = object : ThryveShenAIEventListener {
        override fun onEvent(event: ThryveShenAIEvent) {
            when (event) {
                ThryveShenAIEvent.MEASUREMENT_FINISHED -> { /* Shen AI measurement has finished and is about to be uploaded */}
                ThryveShenAIEvent.MEASUREMENT_FAILED -> { /* Shen AI measurement has failed - handle failure */}
                ThryveShenAIEvent.DATA_UPLOAD_FINISHED -> { /* Shen AI has finished uploading data */}
                ThryveShenAIEvent.DATA_UPLOAD_FAILED -> { /* Shen AI has failed uploading data */}
                ThryveShenAIEvent.USER_SUMMARY_FINISHED -> { /* The Shen AI flow has been completed */ }
            }
        }
    }
 )
    
val thryveSDKConfig = ThryveSDKConfig(
    authId = "ASSIGNED_AUTH_ID",
    authSecret = "ASSIGNED_AUTH_SECRET",
    endUserAlias = "XXXXXXXXXXX",
    endUserId = null, 
    locale = "de",
    bleConfig = bleConfig
)
                 
val thryveSDK = ThryveSDK.getOrCreate(thryveSDKConfig, context)

/**Android SDK 5.0.5 introduced an optional callback to communicate the status 
  of internal processes of getOrCreate. ThryveSDK instance can now be created 
  with a callback as shown in the sample code below.  
**/
val thryveSDK = ThryveSDK.getOrCreate(thryveSDKConfig, context) { thryveResponse ->
       if(thryveResponse.successful){
           //ThyveSDK initialization processes completed successfully
       } else {
           // ThryveSDK initalization process failed. process the ThryveErrors for the specific reason.
           thryveResponse.errors.map { thryveError -> Logger.e(TAG){" getOrCreate ThryveError in onCreate function $thryveError"} }
       }
  }
```

{% endtab %}
{% endtabs %}

<table><thead><tr><th width="150.45184326171875">Parameter</th><th width="419.3529052734375">Description</th><th>Mandatory</th></tr></thead><tbody><tr><td><code>apiKey</code></td><td>The apiKey as provided by ShenAI. Please reach out to our support if you're interested in obtaining an api key for ShenAI.</td><td>yes</td></tr><tr><td><code>eventsListener</code></td><td><p>An object of <code>ThryveShenAIEventListener</code> that will be receiving the following events from the ShenAI module.</p><pre><code>MEASUREMENT_FINISHED
MEASUREMENT_FAILED
DATA_UPLOAD_FINISHED
DATA_UPLOAD_FAILED
USER_FLOW_FINISHED
</code></pre></td><td>no</td></tr></tbody></table>

## Trigger camera-based health data measurement

To record and upload all available camera-based health data use the `measure` function

{% tabs %}
{% tab title="iOS" %}

```swift
import ThryveCore
import ThryveShenAI
import ThryveCommons

...
func measure() {
    ThryveSDK.get().measure { response in
        if response.successful, let success = response.data, success {
            Logger.d { "Camera data Measure: Measurement and data upload finished successfully.")
        } else {
            Logger.d { "Camera data Measure failed: \(String(describing: response.errors?.first?.errorMessage ?? "Unknown error"))")")
        }
    }
}
...
```

{% endtab %}

{% tab title="Android" %}

```kotlin
import com.thryve.sdk.ThryveSDK
import com.thryve.sdk.shenAI

...
fun measure(activity: ComponentActivity){
    ThryveSDK.get()?.measure(activity) {
        runOnUiThread {
            Logger.d(this@MainActivity.TAG) { "Camera data Measure successful = ${it.successful} data = ${it.data}" }
            it.errors.map { error ->
                Logger.d(this@MainActivity.TAG) { "Camera data Measure error $error" }
            }
        }
    }
}
...
```

{% endtab %}

{% tab title="React Native" %}

<pre class="language-javascript"><code class="lang-javascript">import { ThryveSDK } from '@thryve/react-native-sdk';
<strong>
</strong>//measure Camera based Health data
export async function measure() {
  const sdk = new ThryveSDK().getOrCreate(thryvSDKConfig);
  try {
    const response = await sdk.measure();
    if (response?.data) {
      //measure was successfull
    } else {
      //measure failed. process all errors.
      response.errors.forEach((error) => {
        console.log(`Camera Health Data measure error ${error}`);
      });
    }
  } catch (err) {
    console.error('❌ Unexpected error', err);
  }finally {
      sdk.finish();
  }
}
</code></pre>

{% endtab %}

{% tab title="Flutter" %}

```dart
import 'package:thryve_sdk/thryve_sdk.dart';

final class ShenAIPageBloc {
  final ThryveSDK _thryveSDK;
  final Source _dataSource;

  ShenAIPageBloc(this._thryveSDK) : _dataSource = Source.shenAI;

  Future<ThryveResponse<void>> measure() => _thryveSDK.measure(_dataSource.id);

   Future<void> measureCameraHealthData() async {
    final ThryveResponse<void> response = await measure();

    if (!response.isSuccessful) {
      final Iterable<ThryveError> errors = response.errors.whereType<ThryveError>();

      for (final ThryveError error in errors) {
        //process each error here
      }

      throw StateError('Unable to measure health data using ShenAI.');
    }
  }
}
```

{% endtab %}
{% endtabs %}

### Calibration

The ShenAI technology features a calibration option to enhance the accuracy of blood pressure measurements. The feature is triggered using the `calibrate` method of the Thryve SDK. During calibration, users are guided through a process where they provide readings from an external blood pressure monitor. This aids in refining the precision of camera-based measurements. Note that calibration is optional.

{% tabs %}
{% tab title="iOS" %}

```swift
import ThryveCore
import ThryveShenAI
import ThryveCommons

...
func calibrate() {
    ThryveSDK.get().calibrate { response in
        if response.successful, let success = response.data, success {
            Logger.d { "Camera data Calibrate: Measurement and data upload finished successfully.")
        } else {
            Logger.d { "SCamera data Calibrate failed: \(String(describing: response.errors?.first?.errorMessage ?? "Unknown error"))")")
        }
    }
}
...
```

{% endtab %}

{% tab title="Android" %}

```kotlin
import com.thryve.sdk.ThryveSDK
import com.thryve.sdk.shenAI

...
fun calibrate(activity: ComponentActivity){
    ThryveSDK.get()?.calibrate(activity) {
        runOnUiThread {
            Logger.d(this@MainActivity.TAG) { "ShenAI Calibrate successful = ${it.successful} data = ${it.data}" }
            it.errors.map { error ->
                Logger.d(this@MainActivity.TAG) { "ShenAI Calibrate error $error" }
            }
        }
    }
}
...
```

{% endtab %}

{% tab title="React Native" %}
{% hint style="danger" %}
Calibration is not supported in the React Native SDK
{% endhint %}
{% endtab %}

{% tab title="Flutter" %}
{% hint style="danger" %}
Calibration is not supported in the Thryve Flutter SDK
{% endhint %}
{% endtab %}
{% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.thryve.health/integrate-your-mobile-app/camera-based-health-data-recording.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.