XenseAR SDK Installation Guide for Flutter iOS

1. Copy XenseAR SDK

Copy the XenseAR SDK into the following directory:

<flutter root>/ios/unityLibrary/

The SDK package can be downloaded from the Download Page.

2. Add Dependencies

Add the required packages to:

<flutter root>/pubspec.yaml

Add the necessary dependencies:


dependencies:
  # Thêm hai package này vào danh sách
  flutter_embed_unity: ^1.4.0

After adding the dependencies, run the following command in the terminal to download the packages:


flutter pub get

3. Open Xcode Workspace

Open the xcworkspace file at:

<your flutter project>/ios/Runner.xcworkspace

Opening the xcworkspace file instead of the xcodeproj file is critical.
The xcworkspace file is used to combine multiple projects (including the Flutter Runner project and the XenseAR Library).

4. Add XenseAR to Workspace

From the top menu, select File -> Add Files to "Runner"....
Select the xcodeproj file of the XenseAR Library, typically located at:
<your flutter project>/ios/UnityLibrary/Unity-Iphone.xcodeproj
Ensure the XenseAR Library is placed at the same level as Runner.

5. Update Podfile

Edit the file:

<your flutter app>/ios/Podfile

The XenseAR Library requires a minimum deployment target of iOS 13+ to function properly.


# XenseAR Library requires iOS 13+
platform :ios, '13.0'

If the Podfile file is not present, re-run the Flutter iOS setup to regenerate the iOS module (e.g., ensure the ios folder is properly initialized).


flutter clean
flutter pub get

Install pod:


cd ios & pod install

6. Copy Frameworks

Select the Runner project.
Navigate to Target -> Runner -> General -> Frameworks, Libraries, and Embedded Content.
Copy the frameworks from Unity-iPhone/Frameworks/Silver Tau/ into Frameworks, Libraries, and Embedded Content.
Set the Embed option to Embed & Sign.

7. Update Info.plist Permissions

Select the Runner project.
Navigate to Target -> Runner -> Info -> Custom iOS Target Properties.
Add the following keys:
- Privacy - Camera Usage Description
- Privacy - Microphone Usage Description

7. Fix Black Screen Issue

Due to a conflict between Flutter’s rendering lifecycle and a package inside the XenseAR Library, additional code must be added to:

ios/Runner/AppDelegate.swift

Listen for window changes inside the application lifecycle.
Attach the WebView to the UnityView.


import Flutter
import UIKit
 
@main
@objc class AppDelegate: FlutterAppDelegate {
 
    override func application(
        _ application: UIApplication,
        didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
    ) -> Bool {
        GeneratedPluginRegistrant.register(with: self)
 
        // 1️⃣ Observe when new windows appear or become active
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(onWindowChange),
            name: UIWindow.didBecomeVisibleNotification,
            object: nil
        )
 
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(onWindowChange),
            name: UIWindow.didBecomeKeyNotification,
            object: nil
        )
 
        return super.application(application, didFinishLaunchingWithOptions: launchOptions)
    }
 
    // 🔁 Called when any window becomes visible or key
    @objc func onWindowChange(_ notification: Notification) {
        print("🪟 Window change detected — re-checking WKWebView position.")
        // Delay slightly to let UIKit attach new layers
        DispatchQueue.main.async {
            self.moveWKWebViewToFrontmostWindow()
        }
    }
 
    func moveWKWebViewToFrontmostWindow() {
        // 1️⃣ Collect all visible windows from active scenes
        let allWindows = UIApplication.shared.connectedScenes
            .compactMap { ($0 as? UIWindowScene)?.windows }
            .flatMap { $0 }
            .filter { !$0.isHidden } // ignore hidden windows
 
        guard !allWindows.isEmpty else {
            print("❌ No visible windows found.")
            return
        }
 
        // 2️⃣ Locate the UnityView inside any visible window
        guard
            let unityView = allWindows
                .compactMap({ findSubview(in: $0, nameContains: "UnityView") })
                .first
        else {
            print("⚠️ UnityView not found in any window.")
            return
        }
 
        // 3️⃣ Locate the Vuplex fullscreen WKWebView
        guard
            let wkWebView = allWindows
                .compactMap({ findSubview(in: $0, nameContains: "VXFullScreenWKWebView") })
                .first
        else {
            print("⚠️ WKWebView not found in any window.")
            return
        }
 
        // 4️⃣ If WKWebView is already in the UnityView, no action needed
        if wkWebView.superview === unityView {
            print("✅ WKWebView is already inside the UnityView.")
            return
        }
 
        // 5️⃣ Move WKWebView from its current parent to UnityView
        let previousSuperview = wkWebView.superview
        wkWebView.removeFromSuperview()
        unityView.addSubview(wkWebView)
 
        // 6️⃣ Make WKWebView fill the UnityView entirely
        wkWebView.translatesAutoresizingMaskIntoConstraints = false
        NSLayoutConstraint.activate([
            wkWebView.leadingAnchor.constraint(equalTo: unityView.leadingAnchor),
            wkWebView.trailingAnchor.constraint(equalTo: unityView.trailingAnchor),
            wkWebView.topAnchor.constraint(equalTo: unityView.topAnchor),
            wkWebView.bottomAnchor.constraint(equalTo: unityView.bottomAnchor),
        ])
 
        // 7️⃣ Hide and clean up the old window if it’s different from UnityView's window
        if let prevWindow = previousSuperview?.window,
           prevWindow !== unityView.window {
            prevWindow.isHidden = true
        }
 
        // 8️⃣ Remove window observers (no longer needed)
        NotificationCenter.default.removeObserver(
            self,
            name: UIWindow.didBecomeKeyNotification,
            object: nil
        )
        NotificationCenter.default.removeObserver(
            self,
            name: UIWindow.didBecomeVisibleNotification,
            object: nil
        )
 
        print("✅ WKWebView moved from \(String(describing: previousSuperview)) to UnityView \(unityView).")
    }
 
    private func findSubview(in view: UIView, nameContains keyword: String) -> UIView? {
        if String(describing: type(of: view)).contains(keyword) {
            return view
        }
        for subview in view.subviews {
            if let match = findSubview(in: subview, nameContains: keyword) {
                return match
            }
        }
        return nil
    }
}