Data cleansing 3 Use of OpenCV and preprocessing of image data

Aidemy 2020/9/23

Introduction

Hello, it is Yope! I'm a liberal arts college student, but I'm interested in the AI field, so I'm studying at the AI-specialized school "Aidemy". I would like to share the knowledge gained here with you, and I am summarizing it on Qiita. I am very happy that many people have read the previous summary article. Thank you! This is the third post of data cleansing. Nice to meet you.

What to learn this time ・ About image data ・ What you can do with OpenCV (library that handles images)

1 About image data

"Color" on the computer

-In the data, the color is expressed as RGB data in red, green, and blue (255,0,255, etc.). -The image is made up of a collection of points __ (pixels) __. -The number of color elements to represent one pixel is called channel number (“3” because there are three colors in RGB).

Type of image data

・ BMP cannot be compressed and is large in size ・ JPG Highly compressed but not decompressable -PNG compression / decompression possible, transparent processing possible ・ Supports GIF animation, transparent processing possible

2 OpenCV

Load and display images with OpenCV

-OpenCV is a library used when handling images. Import and use cv2. -Read with __cv2.imread ("filename") __ __cv2.imshow ("window name", read image data) Display as __ (window name can be freely assigned)

import cv2
#sample.Load an image with the file name jpg
img = cv2.imread("./4050_data_cleansing_data/sample.jpg ")
#Output img with window name as window
cv2.imshow("window",img)

Creating and saving images (single color)

-To create an image, use the _np.array () function __ (a function that creates a NumPy matrix). In addition, range () is looped multiple times without variables to set vertical and horizontal pixel information (the order is blue-green-red). __ · np.array ([[[B, G, R values] for _ in range (horizontal size)] for in range (vertical size)], dtype = "uint8") __

#(512*512)One side green(0,255,0)Create an image of
img = np.array([[[0,255,0]for _ in range(512)]for _ in range(512)],dtype="uint8")
#Save img(The file name is "green".img」)
cv2.imwrite("green.img",img)

Trimming (cutting), resizing (enlargement / reduction)

-Trimming is __ image data [y-axis start point: y-axis end point, x-axis start point: x-axis end point] __ 0 is the upper left -Resize is changed with __cv2.resize (image data, (width, height)) __.

#Get image size for cropping(height,width,Number of colors)
size = img.shape #(1000,667,3)
#Height is 1/2, width 1/Trimming to 3 (do not specify the start point, divide so that there is no remainder)
new_img=img[:size[0]//2,:size[1]//3]
#Resized to double height and triple width
new_img=cv2.resize(new_img,(new_img.shape[0]*2,new_img.shape[1]*3))

Rotation / reversal

-The rotation of the image requires a transformation called affine transformation, and the transformation matrix must be obtained. -Get the transformation matrix with __cv2.getRotationMatrix2D (center coordinates of image, rotation angle, scaling magnification) __. -And actually rotate with __cv2.warpAffine (image, transformation matrix, output size) __.

-Invert is done with __cv2.flip (image, flipCode) __. For the second argument, if "0" is specified, it will be inverted vertically, if a positive number is specified, it will be inverted horizontally, and if a negative number is specified, it will be inverted vertically and horizontally.

#Get the transformation matrix(Rotate 90 degrees at 2x magnification)
mat=cv2.getRotationMatrix(tuple(np.array([img.shape[1],img.shape[0]])/2),90,2.0)
#Actually rotate
cv2.warpAffine(img,mat,img.shape[::-1][1:3])

Color conversion / color inversion

-Until now, only "BGR color space" was used, but there are other color spaces on OpenCV. -You can change the color space with __cv2.cvtColor (image, conversion code) __. The conversion code is, for example, "cv2.COLOR_BGR2LAB" when you want to convert the BGR color space to a color space called Lab.

-Color inversion can also be done by taking out each pixel in turn and setting the value x to "255-x". You can easily flip it with __cv2.bitwise_not (image) __.

#Change the color space of img to Lab
c_img=cv2.cvtColor(img,cv2.COLOR_BGR2LAB)
#Invert img color
r_img=cv2.bitwise_not(img)

Application of OpenCV

Threshold processing

-In order to reduce the size of the image, color data should be limited to "white" and "black". -__Cv2.threshold (image, threshold, maximum value (concentration), type of threshold processing) __

#Threshold 127, maximum value 255, type cv2.THRESH_TOZERO(0 below the threshold, no change above the threshold)
new_img=cv2.threshold(img,127,255,cv2.THRESH_TOZERO)

masking

-If a black and white mask image is prepared separately and combined with the original image using the cv2.bitwise_and () function, only the white part of the mask image will be output from the original image. This is called "masking". -__Cv2.bitwise_and (original image 1, original image 2 (used when masking), mask = black and white image for mask) __

#Read the mask image (black and white image with 1 channel), resize it, and make it the same size.
mask=cv2.imread("./4050_cleansing_data/mask.png ", 0)
mask=cv2.resize(mask,(img.shape[1],img.shape[0]))
#Masking
cv2.bitwise_and(img,img,mask=mask)

Blur the image

-Use the "Gaussian filter" to blur the image. Average and blur n * n (n is an odd number) around one pixel. · __Cv2.GaussianBlur (image, (n * n value), standard deviation) __

#(51*51)Blur
new_img=cv2.GaussianBlur(img,(51,51),0)

Noise (rough) removal

・ __Cv2.fastNlMeansDenoisingColored (image) __. If it is not a color image, the function name "Colored" may be omitted. (Nl Means refers to a noise reduction filter called Non-local Means Filter, Denoising means noise reduction)

new_img=cv2.fastNlMeansDenoisingColored(img)

Shrinkage / expansion

-Another method of noise processing is to shrink the image once and then expand it again. This method is often used for "threshold processing" noise removal. -Expand with __cv2.dilate (image, filter) __, shrink with __cv2.erode (image, filter) __.

#Threshold processing of img
new_img=cv2.threshold(img,127,255,cv2.THRESH_BYNARY)
#Filter definition
filter=np.array([[0,1,0],[1,0,1],[0,1,0]],np.uint8)
#Shrink and expand
new_img=cv2.erode(new_img,filter)
new_img=cv2.dilate(new_img,filter)

Summary

-Image data is processed by OpenCV. -In OpenCV, you can read images with __imread () __, output images with __imshow () __, resize with __resize () __, invert with __flip () __, and invert colors with ** bitwise_not **. In addition, you can create images, crop, and rotate. -As an application, use __threshold () __ for threshold processing, ** bitwise_and () ** for masking, and __GaussianBlur () __ for blurring. In addition, noise can be removed.

Recommended Posts

Data cleansing 3 Use of OpenCV and preprocessing of image data
Python application: Data cleansing # 3: Use of OpenCV and preprocessing of image data
Preprocessing of prefecture data
About data preprocessing of systems that use machine learning
Story of image analysis of PDF file and data extraction
Overview of natural language processing and its data preprocessing
Use and integration of "Shodan"
Environment construction of python and opencv
Time series analysis 3 Preprocessing of time series data
Preprocessing of Wikipedia dump files and word-separation of large amounts of data by MeCab
Read the graph image with OpenCV and get the coordinates of the final point of the graph
Separation of design and data in matplotlib
Easy introduction of python3 series and OpenCV3
Data cleansing of open data of the occurrence situation of the Ministry of Health, Labor and Welfare
Python: Preprocessing in machine learning: Handling of missing, outlier, and imbalanced data
Use Pillow to make the image transparent and overlay only part of it
[Kaggle] From data reading to preprocessing and encoding
Multivariate LSTM and data preprocessing in TensorFlow 2.x
Generate and post dummy image data with Django
Smoothing of time series and waveform data 3 methods (smoothing)
python development environment -use of pyenv and virtualenv-
Use of past meteorological data 1 (Display of AMeDAS points)
Wavelet transform of images with PyWavelets and OpenCV
I tried using the image filter of OpenCV
cv2 functions and data types (OpenCV python bindings)
[Python + OpenCV] Whiten the transparent part of the image
What to use for Python stacks and queues (speed comparison of each data structure)
Image of closure
Use data class for data storage of Python 3.7 or higher
Analysis of financial data by pandas and its visualization (2)
Full-width and half-width processing of CSV data in Python
List of Python libraries for data scientists and data engineers
Analysis of financial data by pandas and its visualization (1)
Generate a vertical image of a novel from text data
[OpenCV / Python] I tried image analysis of cells with OpenCV
Image capture / OpenCV speed comparison with and without GPU
Statistical hypothesis test of A/B test and required number of data
Overview and tips of seaborn with statistical data visualization
Ubuntu 20.04 on raspberry pi 4 with OpenCV and use with python
Let's use the open data of "Mamebus" in Python
Rotate and scale the image before cropping [python] [OpenCV]
EP 7 Use List Comprehensions Instead of map and filter
Return the image data with Flask of Python and draw it to the canvas element of HTML
I made a function to crop the image of python openCV, so please use it.