\n
![](\"https:\/\/blogger.googleusercontent.com\/img\/b\/R29vZ2xl\/AVvXsEh96stpV5BvqzxBhk12xYFEg5g-5U4H93DJOId6MEDioi_kPsXWnW3tBwd_kYQfIrcaD4h6QF3cylN62tkiTMit-K6haiB8QSJ2Lnp9ggL_bY_hNwu-FX3HquDmZ98rMYoyug6PxJ0qQQ2_7hlB1BfhZXvh8IFweZJDRbplE0CdwrjjOsMH1GKfC_6hHXo\/s1600\/gen-ai-api-android-meta%20%283%29.png\")
<\/p>\nPosted by Caren Chang – Developer Relations Engineer, Chengji Yan – Software Engineer, Taj Darra – Product Manager<\/em><\/p>\n We are excited to announce a set of on-device GenAI APIs, as part of ML Kit<\/a>, to help you integrate Gemini Nano in your Android apps.<\/p>\n To start, we are releasing 4 new APIs:<\/p>\n GenAI APIs are high level APIs that allow for easy integration, similar to existing ML Kit APIs<\/a>. This means you can expect quality results out of the box without extra effort for prompt engineering or fine tuning for specific use cases.<\/p>\n GenAI APIs run on-device and thus provide the following benefits:<\/p>\n To prevent misuse, we also added safety protection in various layers, including base model training, safety-aware LoRA fine-tuning, input and output classifiers and safety evaluations.<\/p>\n There are 4 main components that make up each of the GenAI APIs.<\/p>\n Together, these components make up the high-level GenAI APIs that simplify the effort needed to integrate Gemini Nano in your Android app.<\/p>\n For each API, we formulate a benchmark score based on the evaluation pipeline mentioned above. This score is based on attributes specific to a task. For example, when evaluating the summarization task, one of the attributes we look at is \u201cgrounding\u201d (ie: factual consistency of generated summary with source content).<\/p>\n To provide out-of-box quality for GenAI APIs, we applied feature specific fine-tuning on top of the Gemini Nano base model. This resulted in an increase for the benchmark score of each API as shown below:<\/p>\n In addition, this is a quick reference of how the APIs perform on a Pixel 9 Pro:<\/p>\n This is an example of implementing the GenAI Summarization API to get a one-bullet summary of an article:<\/p>\n <\/p>\n For more examples of implementing the GenAI APIs, check out the official documentation<\/a> and samples on GitHub:<\/p>\n Here is some guidance on how to best use the current GenAI APIs:<\/p>\n For Summarization<\/b>, consider:<\/p>\n For Proofreading<\/b> and Rewriting<\/b> APIs, consider utilizing them during the content creation process for short content below 256 tokens to help with tasks such as:<\/p>\n For the Image Description<\/b> API, consider it for:<\/p>\n Envision is an app that verbalizes the visual world to help people who are blind or have low vision lead more independent lives. A common use case in the app is for users to take a picture to have a document read out loud. Utilizing the GenAI Summarization API, Envision is now able to get a concise summary of a captured document. This significantly enhances the user experience by allowing them to quickly grasp the main points of documents and determine if a more detailed reading is desired, saving them time and effort.<\/p>\n <\/image><\/p>\n GenAI APIs are available on Android devices using optimized MediaTek Dimensity, Qualcomm Snapdragon, and Google Tensor platforms through AICore. For a comprehensive list of devices that support GenAI APIs, refer to our official documentation<\/a>.<\/p>\n Start implementing GenAI APIs in your Android apps today with guidance from our official documentation<\/a> and samples on GitHub: AI Catalog GenAI API Samples with Compose, ML Kit GenAI APIs Quickstart<\/a>.<\/p>\n<\/div>\n![](\"https:\/\/blogger.googleusercontent.com\/img\/b\/R29vZ2xl\/AVvXsEjUnecK6GCD0zX60dSKM1fspl7RcCZh3kaWG-PYKw3Yf4Vyf_3NXlsQoRAVQ_milOQqfBS6gtDODFwK5b9SskqFT7tu6fYDr8WcqxuJQxqIrA9L8VVVsxgSGJogtFqgFe0wVZswvVjTU1Zuf1-ZkAT_bPXL29iIVooQzoStIDZEbaVA0ygvoggMoMsCXrM\/s1600\/gen-ai-api-android-hero%20%281%29.png\")
<\/a><\/p>\n\n
\n
Key benefits of GenAI APIs<\/span><\/h2>\n
\n
\n
How GenAI APIs are built<\/span><\/h2>\n
\n
Evaluating quality of GenAI APIs<\/span><\/h2>\n
\n\n
\n Use case in English<\/b><\/th>\n Gemini Nano Base Model<\/th>\n ML Kit GenAI API<\/th>\n<\/tr>\n \n Summarization<\/td>\n 77.2<\/td>\n 92.1<\/td>\n<\/tr>\n \n Proofreading<\/td>\n 84.3<\/td>\n 90.2<\/td>\n<\/tr>\n \n Rewriting<\/td>\n 79.5<\/td>\n 84.1<\/td>\n<\/tr>\n \n Image Description<\/td>\n 86.9<\/td>\n 92.3<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n\n
\n \n Prefix Speed
(input processing rate)<\/b><\/th>\nDecode Speed
(output generation rate)<\/b><\/th>\n<\/tr>\n\n Text-to-text<\/td>\n 510 tokens\/second<\/td>\n 11 tokens\/second<\/td>\n<\/tr>\n \n Image-to-text<\/td>\n 510 tokens\/second + 0.8 seconds for image encoding<\/td>\n 11 tokens\/second<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Sample usage<\/span><\/h2>\n
val<\/span> articleToSummarize = \"We are excited to announce a set of on-device generative AI APIs...\"<\/span>\n\n\/\/ Define task with desired input and output format<\/span>\nval<\/span> summarizerOptions = SummarizerOptions.builder(context)\n .setInputType(InputType.ARTICLE)\n .setOutputType(OutputType.ONE_BULLET)\n .setLanguage(Language.ENGLISH)\n .build()\nval<\/span> summarizer = Summarization.getClient(summarizerOptions)\n\nsuspend fun<\/span> prepareAndStartSummarization<\/span>(context: Context) {\n \/\/ Check feature availability. Status will be one of the following: <\/span>\n \/\/ UNAVAILABLE, DOWNLOADABLE, DOWNLOADING, AVAILABLE<\/span>\n val<\/span> featureStatus = summarizer.checkFeatureStatus().await()\n\n if<\/span> (featureStatus == FeatureStatus.DOWNLOADABLE) {\n \/\/ Download feature if necessary.<\/span>\n \/\/ If downloadFeature is not called, the first inference request will <\/span>\n \/\/ also trigger the feature to be downloaded if it's not already<\/span>\n \/\/ downloaded.<\/span>\n summarizer.downloadFeature(object : DownloadCallback {\n override<\/span> fun<\/span> onDownloadStarted<\/span>(bytesToDownload: Long) { }\n\n override<\/span> fun<\/span> onDownloadFailed<\/span>(e: GenAiException) { }\n\n override<\/span> fun<\/span> onDownloadProgress<\/span>(totalBytesDownloaded: Long) {}\n\n override<\/span> fun<\/span> onDownloadCompleted<\/span>() {\n startSummarizationRequest(articleToSummarize, summarizer)\n }\n }) \n } else<\/span> if<\/span> (featureStatus == FeatureStatus.DOWNLOADING) {\n \/\/ Inference request will automatically run once feature is <\/span>\n \/\/ downloaded.<\/span>\n \/\/ If Gemini Nano is already downloaded on the device, the <\/span>\n \/\/ feature-specific LoRA adapter model will be downloaded very <\/span>\n \/\/ quickly. However, if Gemini Nano is not already downloaded, <\/span>\n \/\/ the download process may take longer.<\/span>\n startSummarizationRequest(articleToSummarize, summarizer)\n } else<\/span> if<\/span> (featureStatus == FeatureStatus.AVAILABLE) {\n startSummarizationRequest(articleToSummarize, summarizer)\n } \n}\n\nfun<\/span> startSummarizationRequest<\/span>(text: String, summarizer: Summarizer) {\n \/\/ Create task request <\/span>\n val<\/span> summarizationRequest = SummarizationRequest.builder(text).build()\n\n \/\/ Start summarization request with streaming response<\/span>\n summarizer.runInference(summarizationRequest) { newText -> \n \/\/ Show new text in UI<\/span>\n }\n\n \/\/ You can also get a non-streaming response from the request<\/span>\n \/\/ val summarizationResult = summarizer.runInference(summarizationRequest)<\/span>\n \/\/ val summary = summarizationResult.get().summary<\/span>\n}\n\n\/\/ Be sure to release the resource when no longer needed<\/span>\n\/\/ For example, on viewModel.onCleared() or activity.onDestroy()<\/span>\nsummarizer.close()\n<\/pre>\n<\/div>\nUse cases<\/span><\/h2>\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
GenAI API in production<\/span><\/h2>\n
<\/div>\nSupported devices<\/span><\/h2>\n
Learn more<\/span><\/h2>\n