[Image recognition] How to read the result of automatic annotation with VoTT

Introduction

Annotation (teacher label creation) is the most important but troublesome part of machine learning. For image recognition, annotation tools such as VoTT are available, but it is still troublesome to click with the mouse. It would be fun if we could create labels automatically to some extent and fix only the strange parts.

Active Learning Function (VoTT)

In fact, there are functions in the world that can fulfill such demands, such as VoTT's Active Learning function. Introduction of Active Learning function of VoTT --Qiita

However, the trained models that can be used are fixed, and some tasks may not be available. I and others want to Annotate the Marufuku sign, but ** the existing model for object detection is refreshing. I mean, that's why I'm trying to learn the model myself. ** **

So, I will show you how to convert labels created using your favorite logic and model so that they can be read by annotation tools. For example, it is possible to read the following labels created with OpenCV with an annotation tool, modify only the necessary parts, and use them for deep learning.

image.png OpenCV shape detection - PyImageSearch

Verification environment

VoTT 2.1.0 was released as of May 2020 when I wrote the article, but it seemed difficult to load the annotations I made because it became too sophisticated and the structure of the project was complicated. Therefore, this time we will load it with the older version 1.7.2.

procedure

VoTT installation

Use VoTT 1.7.2. Download and install vott-win.exe. Release 1.7.2 · microsoft/VoTT

Data preparation

Create a folder in a suitable location and put the images (JPG or PNG) you want to annotate directly under it. Here, the full path of the folder is C: \ foo \ bar \ myproj. For VoTT 1.7.2, the name of the corresponding project file is C: \ foo \ bar \ myproj.json.

Creating automatic annotation results

make_vott_project.py


import sys
import json
import hashlib
import urllib
from pathlib import Path
from PIL import Image # pillow 7.1.2

imgdir = Path(sys.argv[1]).resolve()
projfile = imgdir.with_suffix(".json")

if projfile.exists():
    #Load an existing project
    f = open(projfile, "r+")
    data = json.load(f)
else:
    #Create a new project
    f = open(projfile, "w")
    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#0cc7ff"] #Area color (optional)
    }

with f:
    for imgfile in imgdir.glob("*.*"):
        if imgfile.suffix.lower() in [".jpg ", ".png "]:
            if imgfile.name in data["frames"]:
                #Skip if there is an entry
                continue
            else:
                #If there is no entry, create a new one
                frame = []
                data["frames"][imgfile.name] = frame

            #Get the size of the image
            img = Image.open(imgfile)
            w, h = img.size
            img.close()

            #List the areas you have detected (provisional)
            points = [ #List the vertices in the order in which they are connected by edges
                {"x": 0.0, "y": 0.0},
                {"x": w,   "y": 0.0},
                {"x": w,   "y": h},
                {"x": 0.0, "y": h}
            ]
            box = {  #Circumscribed rectangle
                "x1": min(p["x"] for p in points),
                "y1": min(p["y"] for p in points),
                "x2": max(p["x"] for p in points),
                "y2": max(p["y"] for p in points)
            }
            region = box.copy()
            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk"], #Tag list to be given to the area
            })
            frame.append(region)

    #Save project
    f.seek(0)
    json.dump(data, f)

If you execute the following from the command prompt, the tag mrfk will be added to the entire area of each image.

python make_vott_project.py C:\foo\bar\myproj

How to edit with VoTT

Launch VoTT 1.7.2, click the image icon and select the C: \ foo \ bar \ myproj folder. ** Please note that the project file name is automatically determined from the folder name. ** ** image.png

The following screen is OK with Continue as it is image.png

A label is attached to the entire image. You can switch the image with the buttons of 2 left-pointing triangles and 2 right-pointing triangles. image.png

If you want to modify the label, select Regions Manipulation in the upper left, then drag and drop the four corners of the area. image.png

You can overwrite and save by selecting File → Save from the menu. You can read the updated json file and create training data appropriately.

Customize

Load annotations created externally such as OpenCV

#List the areas you have detected (provisional)

In the part of, enter the detection result in points. Specify the coordinates of each vertex of the area in the order in which they are connected by edges.

For example, if you want to use the code from the tutorial below OpenCV shape detection - PyImageSearch

frame = []
# loop over the contours
for c in cnts:
    (Abbreviation)
    points = [{"x": p[0], "y": p[1]} for p in c]
    (Abbreviation)
    frame.append(region)

You can make it like this.

Specify the area of the polygon

If you want to specify a complex area that is not a rectangle

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk"], #Tag list to be given to the area
            })

This part

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "polygon", #Change here
                "tags": ["mrfk"], #Tag list to be given to the area
            })

Just change to. This way, when you move one vertex, the other vertices will not move. You can create a non-rectangular shape as shown below. image.png

Make multiple tags

First, specify the name as a comma-separated string in place of `ʻinputTags`` below.

    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk,chst,wide", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#0cc7ff"] #Area color (optional)
    }

For tags in each area, specify a list of tags (a list of Python, not a comma-separated string).

            region.update({
                "width": w,
                "height": h,
                "box": box,
                "points": points,
                "type": "rect",
                "tags": ["mrfk", "chst"], #Tag list to be given to the area
            })

As you can see, it is reflected in the bottom left of the window and in the tooltip of the image. image.png

If you want to further subdivide the existing object detection result class, you can increase only `ʻinputTags`` and edit the class (tag) of each area with VoTT.

Change the color of the tag

If you want to change the color of the area or the text color of the tag list at the bottom left, you can specify the color list in tag_colors below.

    data = {
        "frames": {},
        "framerate": "1",
        "inputTags": "mrfk,chst,wide", #Tag list (separated by commas)
        "suggestionType": "track",
        "scd": False,
        "visitedFrames": [],
        "tag_colors": ["#ff4040", "#ffff40", "#008000"] #Area color (optional)
    }

You will be able to put out in your favorite color as follows. (The image is after editing the annotation by myself) image.png

Recommended Posts

[Image recognition] How to read the result of automatic annotation with VoTT
How to crop the lower right part of the image with Python OpenCV
The result of making the first thing that works with Python (image recognition)
I tried to find the entropy of the image with python
How to enable Read / Write of net.Conn with context with golang
How to read the SNLI dataset
How to get the ID of Type2Tag NXP NTAG213 with nfcpy
How to make a command to read the configuration file with pyramid
Consider the speed of processing to shift the image buffer with numpy.ndarray
[Linux] How to disable the automatic update of the /etc/resolv.conf file (AmazonLinux2)
How to output the output result of the Linux man command to a file
How to monitor the execution status of sqlldr with the pv command
How to check the version of Django
How to read problem data with paiza
Try to image the elevation data of the Geographical Survey Institute with Python
[Introduction to Python] How to sort the contents of a list efficiently with list sort
Read the graph image with OpenCV and get the coordinates of the final point of the graph
Write the result of keyword search with ebaysdk to Google Spread Sheets
I want to stop the automatic deletion of the tmp area with RHEL7
How to query BigQuery with Kubeflow Pipelines and save the result and notes
How to pass the execution result of a shell command in a list in Python
How to calculate the volatility of a brand
How to read a CSV file with Python 2/3
How to find the area of the Voronoi diagram
I tried image recognition of CIFAR-10 with Keras-Learning-
Crop the image to rounded corners with pythonista
How to code a drone using image recognition
How to specify the NIC to scan with amazon-dash
I tried image recognition of CIFAR-10 with Keras-Image recognition-
[Python] How to read excel file with pandas
How to crop an image with Python + OpenCV
How to try the friends-of-friends algorithm with pyfof
I tried to correct the keystone of the image
How to read an array with Python's ConfigParser
How to specify attributes with Mock of python
How to implement "named_scope" of RubyOnRails with Django
How to display in the entire window when setting the background image with tkinter
Checklist on how to avoid turning the elements of numpy's array with for
Note: How to get the last day of the month with python (added the first day of the month)
How to Learn Kaldi with the JUST Corpus
How to get a list of files in the same directory with python
[Introduction to Python] How to get the index of data with a for statement
How to read original data or external data on the Internet with scikit-learn instead of attached data set such as iris
The background of the characters in the text image is overexposed to make it easier to read.
How to identify the element with the smallest number of characters in a Python list?
How to get started with Visual Studio Online ~ The end of the environment construction era ~
[Note] How to write QR code and description in the same image with python
A memo on how to overcome the difficult problem of capturing FX with AI
How to count the number of occurrences of each element in the list in Python with weight
How to change the generated image of GAN to a high quality one to your liking
The 15th offline real-time I tried to solve the problem of how to write with python
Read the Python-Markdown source: How to create a parser
How to know the port number of the xinetd service
Extract the table of image files with OneDrive & Python
How to delete the specified string with the sed command! !! !!
How to get the number of digits in Python
How to scrape image data from flickr with python
I want to grep the execution result of strace
Add information to the bottom of the figure with Matplotlib
[Introduction to Python] How to iterate with the range function?
How to create a submenu with the [Blender] plugin