How to make an artificial intelligence LINE bot with Flask + LINE Messaging API

How to make an artificial intelligence LINE bot with Flask + LINE Messaging API

Nakano Hitoshi's book <a target="_blank" href="https://www.amazon.co.jp/gp/product/B07QLGKJ8T/ref=as_li_tl?ie=UTF8&camp=247&creative=1211&creativeASIN=B07QLGKJ8T&linkCode=as2&tag=kokkahasan-22&linkId=765a3cca2 > Introduction to artificial intelligence development made with Python + LINE --How to make an artificial intelligence LINE bot with Flask + LINE Messaging API <img src = "// ir-jp.amazon-adsystem.com/e/ir?t = kokkahasan-22 & l = am2 & o = 9 & a = B07QLGKJ8T "width =" 1 "height =" 1 "border =" 0 "alt =" "style =" border: none! Important; margin: 0px! Important; "/>

It was pretty interesting. At the end of the sample exercise is a bot app that hides faces with stamps by face recognition using Google Vision API. There are many apps like this, but it may be easier to use if you use a LINE bot.

Finally, he is given homework to try to make multiple face recognitions correspond. Only one person is supported in the book app. There is no answer. It's easy. Let's do our best on our own, Mr. Nakano.

I made an answer with a little effort, so for reference.

Face detection of the image sent by the user with Google Vison API and reply of the composite photo hidden by cat.png (supports multiple faces)
#app.py

#Face detection of the image sent by the user with Google Vison API and cat.Reply a composite photo hidden in png(Supports multiple faces)

import io
import os
import base64
import json
import requests
from flask import Flask, request, abort
from PIL import Image #Pillow install pip3 install pillow

from linebot import (
    LineBotApi, WebhookHandler
)
from linebot.exceptions import (
    InvalidSignatureError
)
from linebot.models import (
    MessageEvent, TextMessage, TextSendMessage, ImageMessage, ImageSendMessage
)


#LINE access token and application secret
ACCESS_TOKEN = ''
SECRET = ''
#Google Vision API key
API_KEY = ''

app = Flask(__name__)

line_bot_api = LineBotApi(ACCESS_TOKEN)
handler = WebhookHandler(SECRET)


@app.route('/')
def hello_world():
    return 'Hello World!'


@app.route('/callback',methods=['POST'])
def callback():
    signature = request.headers['X-Line-Signature']

    body = request.get_data(as_text=True)
    app.logger.info("Request body: " + body)

    try:
        handler.handle(body, signature)
    except InvalidSignatureError:
        print("Invalid signature. Please check your channel access token/channel secret.")
        abort(400)
    
    return 'OK'

@handler.add(MessageEvent,message=ImageMessage)
def handle_message(event):
    try:
        message_content = line_bot_api.get_message_content(event.message.id)
        # event.message.Image body data can be read by specifying id
        # message_content.content #Acquired image file body

        image_base64 = base64.b64encode(message_content.content) #Convert image files to base64

        #Create request body (json.dumps()Converted to JSON)
        req_body = json.dumps({
            'requests': [{
                'image': {
                    'content': image_base64.decode('utf-8')
                },
                'features': [{
                    'type': 'FACE_DETECTION',
                    'maxResults': 20,
                }]
            }]
        })
                            #Vision API endpoint ↓
        res = requests.post("https://vision.googleapis.com/v1/images:annotate?key=" + API_KEY, data=req_body)
        #print('res content is' + res.text)

        result = res.json()
    
        vertices = result["responses"][0]["faceAnnotations"]
        #print('The contents of vertices are' + json.dumps(vertices)) 
        ##response content is response.See json.

        if vertices:
            print('I was able to get')
            image_base = Image.open(io.BytesIO(message_content.content))
            for face in vertices:
                corner = face["boundingPoly"]['vertices'][0]
                print('corner is' + json.dumps(corner))
                print('face["boundingPoly"]["vertices"][1]["x"]Is' + json.dumps(face["boundingPoly"]['vertices'][1]["x"]))
                width = face["boundingPoly"]['vertices'][1]["x"] - face["boundingPoly"]['vertices'][0]["x"]
                height = face["boundingPoly"]['vertices'][2]["y"] - face["boundingPoly"]['vertices'][1]["y"]

                image_cover = Image.open('static/cat.png') # cat.png must be an alpha channel image. ValueError:bad transparency mask error
                image_cover = image_cover.resize((width,height))
                image_base.paste(image_cover, (corner['x'],corner['y']), image_cover)
                # Image.paste(im, box=None, mask=None)
                print('for loop end')

            image_base.save('static/' + event.message.id + '.jpg')


        line_bot_api.reply_message(
            event.reply_token,
            ImageSendMessage(
                    original_content_url = "https://hidden-savannah-xxxxx.herokuapp.com/static/" + event.message.id + ".jpg ",
                    preview_image_url = "https://hidden-savannah-xxxxx.herokuapp.com/static/" + event.message.id + ".jpg "
            )
        )

    except:
        line_bot_api.reply_message(
            event.reply_token,
            TextSendMessage(text="I couldn't recognize my face (animals aren't good, only humans. It's tough to have a profile or too close up")
        )


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 5000)))


reference:

Just in case, the response content of print ('vertices content is' + json.dumps (vertices)) on line 80

[
	{
		"boundingPoly": {
			"vertices": [
				{
					"x": 917,
					"y": 318
				},
				{
					"x": 1174,
					"y": 318
				},
				{
					"x": 1174,
					"y": 616
				},
				{
					"x": 917,
					"y": 616
				}
			]
		},
		"fdBoundingPoly": {
			"vertices": [
				{
					"x": 971,
					"y": 396
				},
				{
					"x": 1163,
					"y": 396
				},
				{
					"x": 1163,
					"y": 588
				},
				{
					"x": 971,
					"y": 588
				}
			]
		},
		"landmarks": [
			{
				"type": "LEFT_EYE",
				"position": {
					"x": 1031.1968,
					"y": 456.0161,
					"z": 0.0003030986
				}
			},
			{
				"type": "RIGHT_EYE",
				"position": {
					"x": 1112.0862,
					"y": 460.92987,
					"z": 28.232975
				}
			},
			{
				"type": "LEFT_OF_LEFT_EYEBROW",
				"position": {
					"x": 1008.84607,
					"y": 436.544,
					"z": -1.8571037
				}
			},
			{
				"type": "RIGHT_OF_LEFT_EYEBROW",
				"position": {
					"x": 1060.1007,
					"y": 440.86813,
					"z": -7.585352
				}
			},
			{
				"type": "LEFT_OF_RIGHT_EYEBROW",
				"position": {
					"x": 1095.2485,
					"y": 442.76245,
					"z": 5.0468025
				}
			},
			{
				"type": "RIGHT_OF_RIGHT_EYEBROW",
				"position": {
					"x": 1131.141,
					"y": 444.30832,
					"z": 41.595203
				}
			},
			{
				"type": "MIDPOINT_BETWEEN_EYES",
				"position": {
					"x": 1075.8728,
					"y": 455.9283,
					"z": -1.5975293
				}
			},
			{
				"type": "NOSE_TIP",
				"position": {
					"x": 1080.8457,
					"y": 504.33997,
					"z": -20.247692
				}
			},
			{
				"type": "UPPER_LIP",
				"position": {
					"x": 1071.2343,
					"y": 531.5437,
					"z": -1.6211907
				}
			},
			{
				"type": "LOWER_LIP",
				"position": {
					"x": 1069.6505,
					"y": 551.9242,
					"z": 4.4038887
				}
			},
			{
				"type": "MOUTH_LEFT",
				"position": {
					"x": 1035.7985,
					"y": 538.815,
					"z": 8.222528
				}
			},
			{
				"type": "MOUTH_RIGHT",
				"position": {
					"x": 1101.0676,
					"y": 541.8905,
					"z": 30.981604
				}
			},
			{
				"type": "MOUTH_CENTER",
				"position": {
					"x": 1070.1655,
					"y": 541.40643,
					"z": 4.1978736
				}
			},
			{
				"type": "NOSE_BOTTOM_RIGHT",
				"position": {
					"x": 1092.8889,
					"y": 510.94235,
					"z": 16.238985
				}
			},
			{
				"type": "NOSE_BOTTOM_LEFT",
				"position": {
					"x": 1049.6199,
					"y": 507.50146,
					"z": 0.9902145
				}
			},
			{
				"type": "NOSE_BOTTOM_CENTER",
				"position": {
					"x": 1072.0765,
					"y": 515.82806,
					"z": -2.7877321
				}
			},
			{
				"type": "LEFT_EYE_TOP_BOUNDARY",
				"position": {
					"x": 1037.2472,
					"y": 452.2355,
					"z": -4.3320293
				}
			},
			{
				"type": "LEFT_EYE_RIGHT_CORNER",
				"position": {
					"x": 1047.4124,
					"y": 459.2465,
					"z": 6.317641
				}
			},
			{
				"type": "LEFT_EYE_BOTTOM_BOUNDARY",
				"position": {
					"x": 1030.3141,
					"y": 461.9699,
					"z": 0.34013578
				}
			},
			{
				"type": "LEFT_EYE_LEFT_CORNER",
				"position": {
					"x": 1018.07513,
					"y": 455.93164,
					"z": 2.3924496
				}
			},
			{
				"type": "LEFT_EYE_PUPIL",
				"position": {
					"x": 1034.6456,
					"y": 457.22366,
					"z": -1.4217875
				}
			},
			{
				"type": "RIGHT_EYE_TOP_BOUNDARY",
				"position": {
					"x": 1109.9236,
					"y": 456.6617,
					"z": 21.767094
				}
			},
			{
				"type": "RIGHT_EYE_RIGHT_CORNER",
				"position": {
					"x": 1119.8134,
					"y": 462.12448,
					"z": 38.996845
				}
			},
			{
				"type": "RIGHT_EYE_BOTTOM_BOUNDARY",
				"position": {
					"x": 1110.3936,
					"y": 466.81308,
					"z": 26.98832
				}
			},
			{
				"type": "RIGHT_EYE_LEFT_CORNER",
				"position": {
					"x": 1094.9646,
					"y": 462.28857,
					"z": 22.470396
				}
			},
			{
				"type": "RIGHT_EYE_PUPIL",
				"position": {
					"x": 1109.2263,
					"y": 461.79114,
					"z": 25.238665
				}
			},
			{
				"type": "LEFT_EYEBROW_UPPER_MIDPOINT",
				"position": {
					"x": 1037.4519,
					"y": 429.95596,
					"z": -10.386488
				}
			},
			{
				"type": "RIGHT_EYEBROW_UPPER_MIDPOINT",
				"position": {
					"x": 1116.0272,
					"y": 434.71762,
					"z": 18.003847
				}
			},
			{
				"type": "LEFT_EAR_TRAGION",
				"position": {
					"x": 954.1669,
					"y": 484.3548,
					"z": 76.21559
				}
			},
			{
				"type": "RIGHT_EAR_TRAGION",
				"position": {
					"x": 1119.6852,
					"y": 494.08078,
					"z": 135.9113
				}
			},
			{
				"type": "FOREHEAD_GLABELLA",
				"position": {
					"x": 1078.9543,
					"y": 441.30212,
					"z": -4.084726
				}
			},
			{
				"type": "CHIN_GNATHION",
				"position": {
					"x": 1062.5234,
					"y": 589.9864,
					"z": 16.94458
				}
			},
			{
				"type": "CHIN_LEFT_GONION",
				"position": {
					"x": 968.6994,
					"y": 536.28186,
					"z": 52.295383
				}
			},
			{
				"type": "CHIN_RIGHT_GONION",
				"position": {
					"x": 1117.5015,
					"y": 545.4246,
					"z": 105.74548
				}
			}
		],
		"rollAngle": 4.5907497,
		"panAngle": 19.758451,
		"tiltAngle": -3.1237326,
		"detectionConfidence": 0.91960925,
		"landmarkingConfidence": 0.5607769,
		"joyLikelihood": "VERY_UNLIKELY",
		"sorrowLikelihood": "VERY_UNLIKELY",
		"angerLikelihood": "VERY_UNLIKELY",
		"surpriseLikelihood": "VERY_UNLIKELY",
		"underExposedLikelihood": "VERY_UNLIKELY",
		"blurredLikelihood": "VERY_UNLIKELY",
		"headwearLikelihood": "LIKELY"
	},
・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・

]


It looks like this
GoogleVisionテスト.jpg
Here are some of the books that didn't work: [Deploy Flask app to heroku (bitterly)](https://qiita.com/atomyah/items/91196f5fda95e4b9c7a6)
Reference: [Face detection using Google Cloud Vision in Python](https://vaaaaaanquish.hatenablog.com/?page=1471257396)

Recommended Posts

How to make an artificial intelligence LINE bot with Flask + LINE Messaging API
How to make a Cisco Webex Teams BOT with Flask
Python beginners decided to make a LINE bot with Flask (Flask rough commentary)
I tried to make an image classification BOT by combining TensorFlow Lite and LINE Messaging API
I tried to make "Sakurai-san" a LINE BOT with API Gateway + Lambda
Make a morphological analysis bot loosely with LINE + Flask
Try to make RESTful API with MVC using Flask 1.0.2
How to make an HTTPS server with Go / Gin
How to operate Discord API with Python (bot registration)
[LINE Messaging API] Create parrot return BOT with Python
I made LINE-bot with Python + Flask + ngrok + LINE Messaging API
How to make a slack bot
The world's most easy-to-understand explanation of how to make LINE BOT (3) [Linkage with server with Git]
How to make an interactive LINE BOT 004 (answer the closing date of a listed company)
[LINE Messaging API] Create a BOT that connects with someone with Python
LINE BOT (Messaging API) development with API Gateway and Lambda (Python) [Part 2]
[Python] Quickly create an API with Flask
How to upload with Heroku, Flask, Python, Git (4)
Introduction to Artificial Intelligence with Python 1 "Genetic Algorithm-Theory-"
How to make an embedded Linux device driver (11)
How to make an embedded Linux device driver (8)
How to make an embedded Linux device driver (1)
Introduction to Artificial Intelligence with Python 2 "Genetic Algorithm-Practice-"
How to make an embedded Linux device driver (4)
How to make a dictionary with a hierarchical structure.
How to make an embedded Linux device driver (7)
How to make an embedded Linux device driver (2)
How to crop an image with Python + OpenCV
How to make an embedded Linux device driver (3)
Python + chatwork + google extension = How to make an easy and funny chat BOT
LINE BOT with Python + AWS Lambda + API Gateway
How to read an array with Python's ConfigParser
[Blender x Python] How to make an animation
How to make an embedded Linux device driver (6)
Make a LINE WORKS bot with Amazon Lex
How to make an embedded Linux device driver (5)
How to make an embedded Linux device driver (10)
How to make Linux compatible with Japanese keyboard
How to make an embedded Linux device driver (9)
Explaining how to make LINE BOT in the world's easiest way (2) [Preparing a bot application in a local environment with Django in Python]
How to make an arbitrary DictCursor with PyMySQL and not return None when NULL
[Learning memo] How to make an application with Django ~ Until Hello World is displayed ~
The world's most easy-to-understand explanation of how to make a LINE BOT (1) [Account preparation]
[Learning memo] How to make an application with Django ~ From virtual environment to pushing to github ~
How to create an article from the command line
[Python] How to draw a line graph with Matplotlib
Explain in detail how to make sounds with python
Playing with a user-local artificial intelligence API in Python
How to upload with Heroku, Flask, Python, Git (Part 3)
How to do zero-padding in one line with OpenCV
How to make a shooting game with toio (Part 1)
I tried to implement an artificial perceptron with python
How to make an interactive CLI tool in Golang
How to upload with Heroku, Flask, Python, Git (Part 1)
How to upload with Heroku, Flask, Python, Git (Part 2)
How to make an embedded Linux device driver (12) (Complete)
How to analyze with Google Colaboratory using Kaggle API
I tried to make an OCR application with PySimpleGUI
[Super easy] Let's make a LINE BOT with Python.
Make LINE BOT (Echolalia)
Made "Unofficial Apple Refurbished Product Introduction" BOT with LINE Messaging API (v2) + API Gateway + lambda (python)