Google ML Kit + Flutter: Part 3(Image Labeling)

Certainly! Continuing from the previous parts of our Google ML Kit and Flutter series, Part 3 will focus on Image Labeling using Google ML Kit in a Flutter app. Image labeling is the process of identifying and tagging objects or features in an image.

Prerequisites: Before we start, make sure you have Flutter and Dart installed on your system. If you haven’t gone through Part 1 & Part 2 of this series, it might be helpful to review it for a general understanding of integrating Google ML Kit into a Flutter app.

Key Concepts:

• Image Labeling: Automatically classifies images into categories based on the objects or features they contain.
• Google ML Kit: A machine learning SDK from Google that provides on-device APIs for image labeling, text recognition, face detection, etc.

What We Will Build:

We will create a simple Flutter app that allows users to capture or pick an image from their device. The app will then use Google ML Kit to label the objects in the image and display them.

Implementation:

Step 1: Add the dependencies

Add dependencies to pubspec.yaml file.

dependencies:
  flutter:
    sdk: flutter
  google_ml_kit: ^0.13.0
  image_picker: ^0.8.6

Step 2: Import

import 'package:google_ml_kit/google_ml_kit.dart';
import 'package:image_picker/image_picker.dart';

Step 3: Install the dependencies

Run the below command in the root directory of your app to install dependency.

flutter pub get

Step 4: Initialize ImageLabeler Instance

There are two ways to create instances of ImageLabeler.

a) First way to create an instance of imageLabeler is by using imageLabeler() method of GoogleMlKit.vision

final imageLabeler = GoogleMlKit.vision.imageLabeler(ImageLabelerOptions(confidenceThreshold: 0.5));

b) Second way to create an instance is to directly use ImageLabeler() with the required options.

final imageLabeler =
      ImageLabeler(options: ImageLabelerOptions(confidenceThreshold: 0.5));

Step 5: Picking Image & Processing

Whenever the user picks or captures any image then we will first convert the file picked into InputImage and then will process that image using the processImage() method of faceDetector instance. This instance will return the list of faces.

var file = await picker.pickImage(source: ImageSource.gallery);
final InputImage inputImage = InputImage.fromFile(File(file!.path));
final List<ImageLabel> labels = await imageLabeler.processImage(inputImage);

Step 6: Building UI

We will now implement the main UI for the app. Here’s a breakdown of the components:

• Image Capture: We provide buttons for users to capture an image from the camera or pick one from the gallery.
• Image Display: Once an image is selected, we display it along with the detected labels and their confidence levels.

              Padding(
                padding: const EdgeInsets.all(18.0),
                child: Row(
                  mainAxisAlignment: MainAxisAlignment.spaceAround,
                  children: [
                    IconButton(
                        onPressed: () async {
                          try {
                            var file = await picker.pickImage(
                                source: ImageSource.camera);
                            final InputImage inputImage =
                                InputImage.fromFile(File(file!.path));
                            final List<ImageLabel> labels =
                                await imageLabeler.processImage(inputImage);

                            setState(() {
                              _file = File(file.path);
                              imageLabels = labels;
                            });
                          } catch (e) {
                            log(e.toString());
                          }
                        },
                        icon: const Icon(
                          Icons.camera_alt,
                          size: 50,
                          color: Colors.blue,
                        )),
                    IconButton(
                        onPressed: () async {
                          try {
                            var file = await picker.pickImage(
                                source: ImageSource.gallery);
                            final InputImage inputImage =
                                InputImage.fromFile(File(file!.path));
                            final List<ImageLabel> labels =
                                await imageLabeler.processImage(inputImage);

                            setState(() {
                              _file = File(file.path);
                              imageLabels = labels;
                            });
                          } catch (e) {
                            log(e.toString());
                          }
                        },
                        icon: const Icon(
                          Icons.add_photo_alternate,
                          size: 50,
                          color: Colors.blue,
                        )),
                  ],
                ),
              )

This section of the code provides two buttons using IconButton widgets, allowing the user to either capture an image using the camera or pick one from the gallery. Here's a brief breakdown:

• IconButton for Camera: When the camera button is pressed, the ImagePicker (referred to as picker) captures an image from the camera. The image is then converted into an InputImage for processing by Google ML Kit's ImageLabeler, which identifies objects in the image.
• IconButton for Gallery: Similarly, when the gallery button is pressed, the user can pick an image from the device’s gallery. The image is processed in the same way to detect labels.

In both cases, the identified labels and the image are updated and displayed on the UI using the setState method.

Widget _buildList(List<ImageLabel> labels) {
    if (labels.isEmpty) {
      return const SizedBox();
    }
    return Expanded(
      child: ListView.builder(
          scrollDirection: Axis.vertical,
          padding: const EdgeInsets.all(1.0),
          itemCount: labels.length,
          itemBuilder: (context, i) {
            return _buildRow(labels[i].label, labels[i].confidence.toString(),
                labels[i].index.toString());
          }),
    );
  }

Widget _buildRow(String label, String confidence, String entityID) {
    return ListTile(
      title: Text(
        "\nLabel: $label \nConfidence: $confidence \nEntityID: $entityID",
        style: const TextStyle(fontSize: 20),
      ),
      dense: true,
    );
  }

This section handles the display of the labeled objects detected in the image:

• _buildList: This function generates a scrollable list of labels if any are detected. It uses a ListView.builder to dynamically create the list items, where each item corresponds to a detected label.
• _buildRow: For each label, this function creates a ListTile that displays the label name, the confidence score (which indicates the certainty of the detection), and the entity ID, all styled in a simple text format.

Together, these functions ensure that the detected labels are neatly presented in the app.

Output:

Full Souce Code

Here is the source code that you can clone and use for your application.
GitHub Link: Image Labeling

Please Follow 💙 and clap 👏, if you find the article useful. And get notified of my upcoming articles.

Follow me on Twitter: @imsnehalsingh and Github: Snehal Singh to hear about my projects.