Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find Required Reason API usage in Flutter Engine and create Privacy Manifest #131494

Closed
vashworth opened this issue Jul 28, 2023 · 10 comments · Fixed by flutter/engine#48951 or flutter/engine#49041
Assignees
Labels
e: OS-version specific Affects only some versions of the relevant operating system engine flutter/engine repository. See also e: labels. P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team-ios Owned by iOS platform team triaged-ios Triaged by iOS platform team

Comments

@vashworth
Copy link
Contributor

Introduced in Xcode 15, Apple is now flagging specific APIs as needed a “Required Reason” to use. All usages of “Required Reason” APIs MUST be outlined in a Privacy manifest. A Privacy manifest is a document outlining how your app/SDK collects and uses a user’s data.

This is also time sensitive (https://developer.apple.com/news/?id=z6fu1dcu).

Starting in fall 2023, when you upload a new app or app update to App Store Connect that uses an API (including from third-party SDKs) that requires a reason, you’ll receive a notice if you haven’t provided an approved reason in your app’s privacy manifest. And starting in spring 2024, in order to upload your new app or app update to App Store Connect, you’ll be required to include an approved reason in the app’s privacy manifest which accurately reflects how your app uses the API.

https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api?language=objc
https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_data_use_in_privacy_manifests?language=objc
https://developer.apple.com/videos/play/wwdc2023/10060/

We'll probably want to also add testing to scan for Required Reason APIs, so they get caught when added.

@vashworth vashworth added platform-ios iOS applications specifically engine flutter/engine repository. See also e: labels. e: OS-version specific Affects only some versions of the relevant operating system fyi-engine For the attention of Engine team team-ios Owned by iOS platform team labels Jul 28, 2023
@stuartmorgan stuartmorgan added P1 High-priority issues at the top of the work list triaged-ios Triaged by iOS platform team labels Jul 31, 2023
@stuartmorgan stuartmorgan self-assigned this Jul 31, 2023
@stuartmorgan
Copy link
Contributor

stuartmorgan commented Aug 1, 2023

For the flutter portion of the engine:

I still need to evaluate all the third-party code we link statically.

@chinmaygarde chinmaygarde added triaged-engine Triaged by Engine team and removed fyi-engine For the attention of Engine team labels Aug 7, 2023
@flutter-triage-bot
Copy link

The triaged-engine label is irrelevant if there is no team-engine label or fyi-engine label.

@flutter-triage-bot flutter-triage-bot bot removed the triaged-engine Triaged by Engine team label Aug 8, 2023
@stuartmorgan
Copy link
Contributor

I'm still finishing the audit, but the new 0A2A.1 handles the large open question here about how to handle Dart's File timestamp APIs.

@stuartmorgan
Copy link
Contributor

I've completed the audit of the engine and all of the third-party code it uses. For iOS release builds, we are covered by:

  • File timestamps:
    • C617.1 for app state restoration in FlutterAppDelegate.mm.
    • 0A2A.1 for implementation of the relevant File wrappers.
  • System boot time:
    • 35F9.1 for various event timing and elapsed time calculations.

@jmagman
Copy link
Member

jmagman commented Dec 12, 2023

Thank you so much @stuartmorgan! I'll start on the PrivacyInfo.xcprivacy to be included in the Flutter.framework.

@jmagman
Copy link
Member

jmagman commented Dec 14, 2023

Well flutter/engine#48951 didn't totally work:
Screenshot 2023-12-14 at 10 02 52 AM

I misread the NSPrivacyCollectedDataTypes section, I thought it wasn't needed. Will add an empty one.

@jmagman
Copy link
Member

jmagman commented Dec 14, 2023

I misread the NSPrivacyCollectedDataTypes section, I thought it wasn't needed. Will add an empty one.

Never mind I did include NSPrivacyCollectedDataTypes, the error is complaining about NSPrivacyCollectedDataType.

https://github.com/flutter/engine/blob/a565cea256c7bafeaa0c26c2f1b0d66a52692d02/shell/platform/darwin/ios/framework/PrivacyInfo.xcprivacy#L9-L12

It probably wants an empty array, not an array with an empty dictionary.

	<key>NSPrivacyCollectedDataTypes</key>
-	<array>
-		<dict/>
- 	</array>
+ 	<array/>

And the error goes away, but the Privacy Report PDF is totally blank so...

@jmagman jmagman reopened this Dec 14, 2023
@jmagman
Copy link
Member

jmagman commented Dec 14, 2023

the Privacy Report PDF is totally blank so...

Blank in both Xcode 15.1 beta 3 and Xcode 15.2 beta. I'm going to close this since we aren't getting errors and the blank report seems like an Xcode bug. I'll play around with it more and file more issues if there's anything we can do about that. As far as I know the xcprivacy file is correct based on Stuart's audit.

Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 28, 2023
@jmagman
Copy link
Member

jmagman commented Jan 8, 2024

This is available in 3.18.0-13.0.pre

2ZeroSix pushed a commit to 2ZeroSix/engine that referenced this issue Apr 10, 2024
Create a `PrivacyInfo.xcprivacy` (this name is required) plist and move it to the top-level of the iOS framework bundle.  `NSPrivacyTracking*` and `NSPrivacyCollectedDataTypes` keys are required, but the values are blank.

Apple [now requires](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files) that third-party frameworks must include this manifest to document usage of particular APIs and [how they are used](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api).

> Starting in fall 2023, when you upload a new app or app update to App Store Connect that uses an API (including from third-party SDKs) that requires a reason, you�ll receive a notice if you haven�t provided an approved reason in your app�s privacy manifest. And starting in spring 2024, in order to upload your new app or app update to App Store Connect, you�ll be required to include an approved reason in the app�s privacy manifest which accurately reflects how your app uses the API.

https://developer.apple.com/news/?id=z6fu1dcu

@stuartmorgan [audited](flutter/flutter#131494 (comment)) the engine and third_party:

> * File timestamps:
>   
>   * `C617.1` for app state restoration in `FlutterAppDelegate.mm`.
>   * `0A2A.1` for implementation of the relevant `File` wrappers.
> * System boot time:
>   
>   * `35F9.1` for various event timing and elapsed time calculations.

Note macOS frameworks do not need to declare `NSPrivacyAccessedAPITypes`.
I don't think this will require recipe or conductor codesign changes since this is a file copied as a resource into the framework, just as the modulemap and Info.plist aren't referenced anywhere.

I'm not quite sure how to test this other than letting it build and generate a `Generate Privacy Report` in a Flutter app in Xcode.  There's no where we check that, say, the Info.plist is copied to the right place in Flutter.framework (even in flutter/flutter).  When this rolls into the framework I will add a check to [ios_content_validation_test.dart](https://github.com/flutter/flutter/blob/3da9bc169837d223496439b6d5f6b7e1a82d4318/dev/devicelab/bin/tasks/ios_content_validation_test.dart)

On this PR the `PrivacyInfo.xcprivacy` was written to the expected location in the Flutter.framework:
https://cas-viewer.appspot.com/projects/chromium-swarm/instances/default_instance/blobs/d45cd0809420f08145c7b78ea96cba6e7ea48d8ecfdc8fd2411f82fa65444714/516/tree

Fixes flutter/flutter#131494

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
wlx64133751 pushed a commit to wlx64133751/engine that referenced this issue May 9, 2024
Create a `PrivacyInfo.xcprivacy` (this name is required) plist and move it to the top-level of the iOS framework bundle.  `NSPrivacyTracking*` and `NSPrivacyCollectedDataTypes` keys are required, but the values are blank.

Apple [now requires](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files) that third-party frameworks must include this manifest to document usage of particular APIs and [how they are used](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api).

> Starting in fall 2023, when you upload a new app or app update to App Store Connect that uses an API (including from third-party SDKs) that requires a reason, you�ll receive a notice if you haven�t provided an approved reason in your app�s privacy manifest. And starting in spring 2024, in order to upload your new app or app update to App Store Connect, you�ll be required to include an approved reason in the app�s privacy manifest which accurately reflects how your app uses the API.

https://developer.apple.com/news/?id=z6fu1dcu

@stuartmorgan [audited](flutter/flutter#131494 (comment)) the engine and third_party:

> * File timestamps:
>
>   * `C617.1` for app state restoration in `FlutterAppDelegate.mm`.
>   * `0A2A.1` for implementation of the relevant `File` wrappers.
> * System boot time:
>
>   * `35F9.1` for various event timing and elapsed time calculations.

Note macOS frameworks do not need to declare `NSPrivacyAccessedAPITypes`.
I don't think this will require recipe or conductor codesign changes since this is a file copied as a resource into the framework, just as the modulemap and Info.plist aren't referenced anywhere.

I'm not quite sure how to test this other than letting it build and generate a `Generate Privacy Report` in a Flutter app in Xcode.  There's no where we check that, say, the Info.plist is copied to the right place in Flutter.framework (even in flutter/flutter).  When this rolls into the framework I will add a check to [ios_content_validation_test.dart](https://github.com/flutter/flutter/blob/3da9bc169837d223496439b6d5f6b7e1a82d4318/dev/devicelab/bin/tasks/ios_content_validation_test.dart)

On this PR the `PrivacyInfo.xcprivacy` was written to the expected location in the Flutter.framework:
https://cas-viewer.appspot.com/projects/chromium-swarm/instances/default_instance/blobs/d45cd0809420f08145c7b78ea96cba6e7ea48d8ecfdc8fd2411f82fa65444714/516/tree

Fixes flutter/flutter#131494

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style

(cherry picked from commit cf7c260)
auto-submit bot pushed a commit to flutter/engine that referenced this issue Sep 14, 2024
Create a `PrivacyInfo.xcprivacy` (this name is required) plist and move it to the top-level of the macOS framework bundle.  `NSPrivacyTracking*` and `NSPrivacyCollectedDataTypes` keys are required, but the values are blank.  macOS explicitly does not need the `NSPrivacyAccessedAPITypes` (see more info in flutter/flutter#143381)

You can see on this PR it's copied to the correct path in the framework https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8737163270670636097/+/u/Global_generators/Release-FlutterMacOS.framework/stdout:

```
  adding: FlutterMacOS.xcframework/macos-arm64_x86_64/FlutterMacOS.framework/PrivacyInfo.xcprivacy (deflated 35%)
```

There's no way to test this except to submit a macOS app with this framework to TestFlight.

I can't find a good spot in the engine to validate the structure of the framework output.  I hereby pledge to add a macOS test to the framework post-roll flutter/flutter#155189 � 

iOS framework variant of this PR #48951
Fixes flutter/flutter#131494

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
flutteractionsbot pushed a commit to flutteractionsbot/engine that referenced this issue Sep 23, 2024
Create a `PrivacyInfo.xcprivacy` (this name is required) plist and move it to the top-level of the macOS framework bundle.  `NSPrivacyTracking*` and `NSPrivacyCollectedDataTypes` keys are required, but the values are blank.  macOS explicitly does not need the `NSPrivacyAccessedAPITypes` (see more info in flutter/flutter#143381)

You can see on this PR it's copied to the correct path in the framework https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8737163270670636097/+/u/Global_generators/Release-FlutterMacOS.framework/stdout:

```
  adding: FlutterMacOS.xcframework/macos-arm64_x86_64/FlutterMacOS.framework/PrivacyInfo.xcprivacy (deflated 35%)
```

There's no way to test this except to submit a macOS app with this framework to TestFlight.

I can't find a good spot in the engine to validate the structure of the framework output.  I hereby pledge to add a macOS test to the framework post-roll flutter/flutter#155189 � 

iOS framework variant of this PR flutter#48951
Fixes flutter/flutter#131494

[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
e: OS-version specific Affects only some versions of the relevant operating system engine flutter/engine repository. See also e: labels. P1 High-priority issues at the top of the work list platform-ios iOS applications specifically team-ios Owned by iOS platform team triaged-ios Triaged by iOS platform team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants