1. Overview
|
|
Welcome to the Recommendations with TensorFlow Lite and Firebase codelab. In this codelab you'll learn how to use TensorFlow Lite and Firebase to deploy a recommendation model to your app. This codelab is based on this TensorFlow Lite example.
Recommendations allow apps to use machine learning to intelligently serve the most relevant content for each user. They take into account past user behavior to suggest app's content the user might like to interact with in the future by using a model trained on the aggregate behavior of a large number of other users.
This tutorial shows how to obtain data from your app's users with Firebase Analytics, build a machine learning model for recommendations from that data, and then use that model in an Android app to run inference and obtain recommendations. In particular, our recommendations will suggest which movies a user would most likely watch given the list of movies the user has liked previously.
What you'll learn
- Integrate Firebase Analytics into an android app to collect user behavior data
- Export that data into Google Big Query
- Pre-process the data and train a TF Lite recommendations model
- Deploy the TF Lite model to Firebase ML and access it from your app
- Run on device inference using the model to suggest recommendations to users
What you'll need
- Latest Android Studio version.
- Sample code.
- A test device with Android 7+ and Google Play services 9.8 or later, or an Emulator with Google Play services 9.8 or later
- If using a device, a connection cable.
How will you use this tutorial?
How would rate your experience with building Android apps?
2. Get the sample code
Clone the GitHub repository from the command line.
$ git clone https://github.com/FirebaseExtended/codelab-contentrecommendation-android.git
3. Import the starter app
From Android Studio, select the codelab-recommendations-android directory ( 
) from the sample code download (File > Open > .../codelab-recommendations-android/start).
You should now have the start project open in Android Studio.
4. Create Firebase console project
Create a new project
- Go to the Firebase console.
- Select Add project (or Create a project if it's the first one).
- Select or enter a Project name and click Continue.
- Ensure that "Enable Google Analytics for this project" is enabled.
- Follow the remaining setup steps in the Firebase console, then click Create project (or Add Firebase, if you're using an existing Google project).
5. Add Firebase
- From the overview screen of your new project, click the Android icon to launch the setup workflow.
- Enter the codelab's package name:
com.google.firebase.codelabs.recommendations - Select Register app.
Add google-services.json file to your app
After adding the package name and selecting Register, click Download google-services.json to obtain your Firebase Android config file then copy the google-services.json file into the app directory in your project. After the file is downloaded you can Skip the next steps shown in the console (they've already been done for you in the build-android-start project).
Add google-services plugin to your app
The google-services plugin uses the google-services.json file to configure your application to use Firebase. The following lines should already be added to the build.gradle.kts files in the project (check to confirm):
app/build.grade.kts
plugins {
id("com.google.gms.google-services")
}
build.grade.kts
plugins {
id("com.google.gms.google-services") version "4.3.15" apply false
}
Sync your project with gradle files
To be sure that all dependencies are available to your app, you should sync your project with gradle files at this point. Select File > Sync Project with Gradle Files from the Android Studio toolbar.
6. Run the starter app
Now that you have imported the project into Android Studio and configured the google-services plugin with your JSON file, you are ready to run the app for the first time. Connect your Android device, and click Run ( 
)in the Android Studio toolbar.
The app should launch on your device. At this point, you can see a functioning application that shows a tab with a list of movies, a Liked movies tab, and a Recommendations tab. You can click on a movie in the list of movies to add it to your liked list. After completing the remaining steps of the codelab, we will be able to generate movie recommendations in the Recommendations tab.
7. Add Firebase Analytics to the app
In this step, you will add Firebase Analytics to the app to log user behavior data (in this case, which movies a user likes). This data will be used in aggregate in future steps to train the recommendations model.
Add Firebase Bill of Materials and Analytics dependency
The following dependencies are necessary to add Firebase Analytics to your app. They should already be included in the app/build.gradle.kts file (verify).
app/build.grade.kts
implementation(platform("com.google.firebase:firebase-bom:32.0.0"))
implementation("com.google.firebase:firebase-analytics-ktx")
Set up Firebase Analytics in the app
The LikedMoviesViewModel contains functions to store the movies the user likes. Every time the user likes a new movie, we want to also send off an analytics log event to record that like.
Add the onMovieLiked function with the code below to register an analytics event when the user clicks like on a movie.
LikedMoviesViewModel.kt
import com.google.firebase.analytics.FirebaseAnalytics
import com.google.firebase.analytics.ktx.analytics
import com.google.firebase.analytics.ktx.logEvent
import com.google.firebase.ktx.Firebase
class LikedMoviesViewModel internal constructor (application: Application) : AndroidViewModel(application) {
...
fun onMovieLiked(movie: Movie) {
movies.setLike(movie, true)
logAnalyticsEvent(movie.id.toString())
}
}
Add the following field and function to log an Analytics event when a movie is added to the user's Liked list.
LikedMoviesViewModel.kt
import com.google.firebase.analytics.FirebaseAnalytics
import com.google.firebase.analytics.ktx.analytics
import com.google.firebase.analytics.ktx.logEvent
import com.google.firebase.ktx.Firebase
class LikedMoviesViewModel internal constructor (application: Application) : AndroidViewModel(application) {
...
private val firebaseAnalytics = Firebase.analytics
...
/**
* Logs an event in Firebase Analytics that is used in aggregate to train the recommendations
* model.
*/
private fun logAnalyticsEvent(id: String) {
firebaseAnalytics.logEvent(FirebaseAnalytics.Event.SELECT_ITEM) {
param(FirebaseAnalytics.Param.ITEM_ID, id)
}
}
8. Test your Analytics integration
In this step, we will generate Analytics events in the app and verify that they are being sent to the Firebase Console.
Enable Analytics Debug Logging
Firebase Analytics is designed to maximize user battery life and will batch events on device and only send them to Firebase occasionally. For debugging purposes, we can disable this behavior to see events as they are logged in real time by running the following command in the shell.
Terminal
adb shell setprop debug.firebase.analytics.app com.google.firebase.codelabs.recommendations
Verify Analytics events are generated
- In Android studio, open the Logcat window to examine logging from your app.
- Set the Logcat filter to the string "Logging event".
- Verify that "select_item" Analytics events are emitted every time you like a movie in the app.
At this point, you have successfully integrated Firebase Analytics into your app. As users use your app and like movies, their likes will be logged in aggregate. We will use this aggregate data in the rest of this codelab to train our recommendations model. The following is an optional step to see the same Analytics events you saw in Logcat also stream in to the Firebase console. Feel free to skip to the next page.
Optional: Confirm Analytics events in Firebase Console
- Go to the Firebase console.
- Select DebugView under Analytics
- In Android Studio, select Run to launch the app and add some movies to your Liked list.
- In the Firebase console's DebugView, verify that these events are being logged as you add movies in the app.
9. Export Analytics data to Big Query
Big Query is a Google Cloud product that allows you to examine and process large amounts of data. In this step, you will connect your Firebase Console project to Big Query so that the Analytics data generated by your app is automatically exported to Big Query.
Enable Big Query export
- Go to the Firebase console.
- Select the Settings gear icon next to Project Overview, and then select Project settings
- Select the Integrations tab.
- Select Link (or Manage) inside the BigQuery block.
- Select Next in the About Linking Firebase to BigQuery step.
- Under the Configure integration section, click the switch to enable sending Google Analytics data and select Link to BigQuery.
You have now enabled your Firebase console project to automatically send Firebase Analytics event data to Big Query. This happens automatically without any further interaction, however, the first export that creates the analytics dataset in BigQuery may not happen for 24 hours. After the dataset is created, Firebase continually exports new Analytics events to Big Query into the intraday table, and groups events from past days in the events table.
Training a recommendations model requires a lot of data. Since we don't already have an app generating large amounts of data, in the next step we will import a sample dataset into BigQuery to use for the rest of this tutorial.
10. Use BigQuery to obtain model training data
Now that we have connected our Firebase Console to export to BigQuery, our app analytics event data will automatically show up in the BigQuery console after some time. To get some initial data for the purposes of this tutorial, in this step we will import an existing sample dataset into your BigQuery console to use to train our recommendations model.
Import sample dataset into BigQuery
- Go to the BigQuery dashboard in the Google cloud console.
- Select your project name in the menu.
- Select your project name in the bottom of the BigQuery left navigation to see details.
- Select Create dataset to open the dataset creation panel.
- Enter ‘firebase_recommendations_dataset' for the Dataset ID and select Create dataset.
- The new dataset will show up in the left menu under the project name. Click it.
- Select Create table to open the table creation panel.
- For Create table from select ‘Google Cloud Storage'.
- In the Select file from GCS bucket field, enter ‘gs://firebase-recommendations/recommendations-test/formatted_data_filtered.txt'.
- Select ‘JSONL' in the File format drop down.
- Enter ‘recommendations_table' for the Table name.
- Check the box under Schema > Auto detect > Schema and input parameters
- Select Create table
Explore sample dataset
At this point, you can optionally explore the schema and preview this dataset.
- Select firebase-recommendations-dataset in the left menu to expand the tables it contains.
- Select the recommendations-table table to view the table schema.
- Select Preview to see the actual Analytics event data this table contains.
Create service account credentials
Now, we will create service account credentials in our Google Cloud console project that we can use in the Colab environment in the following step to access and load our BigQuery data.
- Make sure that billing is enabled for your Google Cloud project.
- Enable the BigQuery and BigQuery Storage API APIs. < click here>
- Go to the Create Service Account Key page.
- From the Service account list, select New service account.
- In the Service account name field, enter a name.
- From the Role list, select Project > Owner.
- Click Create. A JSON file that contains your key downloads to your computer.
In the next step, we will use Google Colab to preprocess this data and train our recommendations model.
11. Preprocess data and train recommendations model
In this step, we will use a Colab notebook to perform the following steps:
- import the BigQuery data into the Colab notebook
- preprocess the data to prepare it for model training
- train the recommendations model on the analytics data
- export the model as a TF lite model
- deploy the model to the Firebase Console so we can use it in our app
Before we launch the Colab training notebook, we will first enable the Firebase Model Management API so Colab can deploy the trained model to our Firebase console.
Enable Firebase Model Management API
Create a bucket to store your ML models
In your Firebase Console, go to Storage and click Get started. 
Follow the dialogue to get your bucket set up.
Enable Firebase ML API
Go to Firebase ML API page on Google Cloud Console and click Enable.
Use Colab notebook to train and deploy the model
Open the colab notebook using the following link and complete the steps within. After finishing the steps in the Colab notebook, you will have a TF lite model file deployed to the Firebase console that we can sync down to our app.
Open in Colab
12. Download the model in your app
In this step, we'll modify our app to download the model we just trained from Firebase Machine Learning.
Add Firebase ML dependency
The following dependency is needed in order to use Firebase Machine Learning models in your app. It should already be added (verify).
app/build.grade.kts
implementation("com.google.firebase:firebase-ml-modeldownloader:24.1.2")
Download the model with Firebase Model Manager API
Copy the code below into RecommendationClient.kt to set up the conditions under which model download occurs and create a download task to sync the remote model to our app.
RecommendationClient.kt
private fun downloadModel(modelName: String) {
val conditions = CustomModelDownloadConditions.Builder()
.requireWifi()
.build()
FirebaseModelDownloader.getInstance()
.getModel(modelName, DownloadType.LOCAL_MODEL, conditions)
.addOnCompleteListener {
if (!it.isSuccessful) {
showToast(context, "Failed to get model file.")
} else {
showToast(context, "Downloaded remote model: $modelName")
GlobalScope.launch { initializeInterpreter(it.result) }
}
}
.addOnFailureListener {
showToast(context, "Model download failed for recommendations, please check your connection.")
}
}
13. Integrate the Tensorflow Lite recommendation model in your app
Tensorflow Lite runtime will let you use your model in the app to generate recommendations. In the previous step we initialized a TFlite interpreter with the model file we downloaded. In this step, we'll first load a dictionary and labels to accompany our model in the inference step, then we'll add pre-processing to generate the inputs to our model and post-processing where we will extract the results from our inference.
Load Dictionary and Labels
The labels used to generate the recommendation candidates by the recommendations model are listed in the file sorted_movie_vocab.json in the res/assets folder. Copy the following code to load these candidates.
RecommendationClient.kt
/** Load recommendation candidate list. */
private suspend fun loadCandidateList() {
return withContext(Dispatchers.IO) {
val collection = MovieRepository.getInstance(context).getContent()
for (item in collection) {
candidates[item.id] = item
}
Log.v(TAG, "Candidate list loaded.")
}
}
Implement Pre-processing
In the pre-processing step, we change the form of the input data to match what our model expects. Here, we pad the input length with a placeholder value if we have not generated a lot of user likes already. Copy the code below:
RecommendationClient.kt
/** Given a list of selected items, preprocess to get tflite input. */
@Synchronized
private suspend fun preprocess(selectedMovies: List<Movie>): IntArray {
return withContext(Dispatchers.Default) {
val inputContext = IntArray(config.inputLength)
for (i in 0 until config.inputLength) {
if (i < selectedMovies.size) {
val (id) = selectedMovies[i]
inputContext[i] = id
} else {
// Padding input.
inputContext[i] = config.pad
}
}
inputContext
}
}
Run interpreter to generate recommendations
Here we use the model we downloaded in a previous step to run inference on our pre-processed input. We set the type of input and output for our model and run inference to generate our movie recommendations. Copy the following code into your app.
RecommendationClient.kt
/** Given a list of selected items, and returns the recommendation results. */
@Synchronized
suspend fun recommend(selectedMovies: List<Movie>): List<Result> {
return withContext(Dispatchers.Default) {
val inputs = arrayOf<Any>(preprocess(selectedMovies))
// Run inference.
val outputIds = IntArray(config.outputLength)
val confidences = FloatArray(config.outputLength)
val outputs: MutableMap<Int, Any> = HashMap()
outputs[config.outputIdsIndex] = outputIds
outputs[config.outputScoresIndex] = confidences
tflite?.let {
it.runForMultipleInputsOutputs(inputs, outputs)
postprocess(outputIds, confidences, selectedMovies)
} ?: run {
Log.e(TAG, "No tflite interpreter loaded")
emptyList()
}
}
}
Implement Post-processing
Finally, in this step we post-process the output from our model, selecting the results with the highest confidence and removing contained values (movies the user has already liked). Copy the following code into your app.
RecommendationClient.kt
/** Postprocess to gets results from tflite inference. */
@Synchronized
private suspend fun postprocess(
outputIds: IntArray, confidences: FloatArray, selectedMovies: List<Movie>
): List<Result> {
return withContext(Dispatchers.Default) {
val results = ArrayList<Result>()
// Add recommendation results. Filter null or contained items.
for (i in outputIds.indices) {
if (results.size >= config.topK) {
Log.v(TAG, String.format("Selected top K: %d. Ignore the rest.", config.topK))
break
}
val id = outputIds[i]
val item = candidates[id]
if (item == null) {
Log.v(TAG, String.format("Inference output[%d]. Id: %s is null", i, id))
continue
}
if (selectedMovies.contains(item)) {
Log.v(TAG, String.format("Inference output[%d]. Id: %s is contained", i, id))
continue
}
val result = Result(
id, item,
confidences[i]
)
results.add(result)
Log.v(TAG, String.format("Inference output[%d]. Result: %s", i, result))
}
results
}
}
Test your app!
Re-run your app. As you select a few movies, it should automatically download the new model and start generating recommendations!
14. Congratulations!
You have built a recommendations feature into your app using TensorFlow Lite and Firebase. Note that the techniques and pipeline shown in this codelab can be generalized and used to serve other types of recommendations as well.
What we've covered
- Firebase ML
- Firebase Analytics
- Export analytics events to BigQuery
- Preprocess analytics events
- Train recommendations TensorFlow model
- Export model and deploy to Firebase Console
- Serve movie recommendations in an app
Next Steps
- Implement Firebase ML recommendations in your app.