[iOS] Camera preview is rotating even if the app supports only portrait mode

[alexLipici-tmd]

Describe the bug
I implemented the 1.1.0-beta1 version of Azure Communication Calling and I observed that camera preview rotates even if the app supports only portrait as device orientation. This behaviour happens when the Portrait Orientation Lock is set to off from device settings. There is any setting for locking the rotation of the stream?

Expected behavior
Camera preview shouldn't rotate if the app's Device Orientation is set to Portrait only and Portrait Orientation Lock is set to off.

Smartphone:

• Device: iPhone XR (iOS 13.4), iPhone 8 Plus (iOS 14.4), iPhone 11 Pro Max (iOS 14.5) and maybe others (these was the testing devices)

[raosanat]

Hi @alexLipici-tmd we are able to reproduce the issue at our end. We will see what fix we can do either in the SDK or in the application to fix the problem.

[raosanat]

Hi @alexLipici-tmd We have a fix that you can do in the application . After the CallAgent is created in the completion handler if you call this Api, UIDevice.current.endGeneratingDeviceOrientationNotifications
For e.g.

        self.callClient!.createCallAgent(userCredential: userCredential!,
                                         options: options,
                                         cxproviderConfig: createProviderConfig(),
                                         completionHandler: { (callAgent, error) in
                if error == nil {
                    UIDevice.current.endGeneratingDeviceOrientationNotifications()
                    self.callAgent!.delegate = self
                }
        })

This will prevent from any device orientation change notification that iOS will trigger from being sent to the SDK.
So let me explain why this happens.

1. We register UIDeviceOrientationDidChangeNotification notification to be received when creating CallAgent so that we can rotate the video frames if orientation of the device changes.
2. Event though App sets orientation to Portrait mode only, iOS still sends this notification when orientation changes from Portrait to Landscape.
3. For us there is no way in SDK to know which orientation application has set to so that we can ignore this notification or not register at all.

So the best way for now would be to not let the iOS send the notification , its important that you call the UIDevice API after CallAgent is created, as in the above code snippet.

Let me know if this solution works.
Thanks.

[alexLipici-tmd]

Hi @raosanat! I tried what you suggested and it doesn't change anything. The issue still reproduce. I've tried also moving UIDevice.current.endGeneratingDeviceOrientationNotifications() on AppDelegate, but it's the same.

[raosanat]

Hi @alexLipici-tmd this solution works for us and verified its working. Can you send the snippet of code that you are testing with ?

[xumo-95]

Hi @alexLipici-tmd I also verified that the solution works for me. Can you please provide more information?

[alexLipici-tmd]

Hi @xumo-95. The code snippet looks like this:

`
self.callClient?.createCallAgent(userCredential: tokenCredential, options: options) { [weak self] (agent, error) in
guard let self = self else { return }

        guard error == nil else {
            completionHandler(.failure(error!))
            return
        }

        UIDevice.current.endGeneratingDeviceOrientationNotifications()

        self.callAgent = agent
        completionHandler(.success(()))
    }

`

Also, keep in mind to set the app to support only Portrait mode (Project Settings->General->Device Orientation->check only the Portrait) and set the Portrait Orientation Lock to off from phone settings.

[RinaRish]

AB#2545996

[xumo-95]

Hi @alexLipici-tmd, I made sure that the app supports Portrait only and turned off the Portrait Orientation Lock and the solution still works for me. We will look further into it.

[kevinyulu]

Hi @xumo-95 and @alexLipici-tmd, I am also investigating the issue in the iOS calling composite app. The iOS calling composite app also has this camera origination switch issue for a portrait only screen in that app. Was the fix suggested by @raosanat working for you? I tried in a few places where the iOS screen was created, and the issue was still intermittently reproducible.

iOS issue for reference: https://skype.visualstudio.com/SPOOL/_workitems/edit/2612386

[alexLipici-tmd]

Hi @kevinyulu. The fix suggestion isn't working for me also

[raosanat]

This issue is a combination of bug in our SDK and on some devices. We have fixed this and will be available in our release next week.

[raosanat]

Fixed in the latest release v2.1.0-beta.1.
Please re-open the bug if still reproducible, thanks.

[kevinyulu]

Hi @alexLipici-tmd , would it be possible to have the latest release tested on your side as well? I tried using the iOS app but the issue seems still happening

[raosanat]

@kevinyulu we have tested on iPhone X and iPhone SE with which we were able to reproduce the issue with.
Which device are you using for testing ?

[kevinyulu]

@raosanat, after I tested the app a few more times, I think the issue can be resolved by using the the workaround you suggested:
UIDevice.current.endGeneratingDeviceOrientationNotifications(). In the Composite app our team is building, there is a screen that has only one orientation set by us and app is using camera preview on that screen. The SDK view would not know this orientation variable in the iOS app so I think we'll need to toggle the device orientation change notification on and off for that screen.

[FreakTheMighty]

@raosanat this was pretty painful to work through and find a solution for. We've been working with the new raw media API's on iOS and were finding that the autorotation caused undesirable affects. For raw media I can't really see a good reason to auto-rotate, but at very least this should be documented.

[gaobob-msft]

@FreakTheMighty can you please describe your situation a bit more. Which component is auto rotating? Are you using raw incoming video or raw outgoing video. Are you seeing the incoming video stream auto rotating or the local camera preview?

[FreakTheMighty]

We're using raw outgoing video from iOS to our web app. In particular, we're sending frames from ARKit. The video, as it is received in the browser, shifts resolution from WxH to HxW depending on the orientation. Oddly, they are inverted from what I'd expect. In portrait mode, we get WxH, and in landscape HxW.

It also flips depending on whether your portrait-up vs portrait-down.

For our use case we really don't want any autor rotation. If we send a frame with a particular image coordinate frame we want to receive it in that same frame.

[gaobob-msft]

Thanks @FreakTheMighty! We'll take a look on our side.

[FreakTheMighty]

@gaobob-msft any update on this? We're finding endGeneratingDeviceOrientationNotifications is unreliable, sometimes it still rotates. To make things worse, if we don't call it, the rotation behaves very oddly. Video shows up in landscape when the device is in portrait and in portrait when the device is in landscape.

We've used the raw media examples as a starting place.

[gaobob-msft]

@FreakTheMighty Yes we are current working on solving this, hoping to have something in early May.

[FreakTheMighty]

@gaobob-msft we've found that if we call
await UIDevice.current.endGeneratingDeviceOrientationNotifications twice, then it stops the video from rotating. Our problem now is that we have no way to get the device orientation ourselves.

How are those fixes coming?

[gaobob-msft]

@FreakTheMighty we released the latest SDK with all the raw media fixed, can you please try to see if it resolves your issue. https://github.com/Azure/Communication/releases