[Note] Based on the latitude and longitude of the CSV file, we created a script that extracts data in the target range and adds a mesh code.

Purpose

Based on the latitude and longitude of the CSV file, I created a script that extracts data in the target range and adds a mesh code.

Prerequisites

[Note] I created a script to convert grib2 format data to a csv file

--Csv file to enter ――The 5th and 6th columns are the longitude and latitude, and the 7th column is the precipitation value. https___qiita-image-store.s3.ap-northeast-1.amazonaws.com_0_205045_ccbdb00a-4927-32b6-4a24-72c7589f6f40.png

Run

--Specify the input folder (assuming there are multiple csv files) and the output CSV file. --Specify the target range for data extraction in latitude and longitude (y1, y2, x1, x2). ――This article covers the Kanto area. --Run the Python script.

highres_nowcast_add_meshcode.py


# -*- coding: utf-8 -*-
import os
import glob
import csv
import pandas as pd
import pprint
import calendar
import jismesh.utils as ju

#Specifying the output file
output_csvfile = "./out/highres_nowcast_anl_kanto.csv"

#Target range setting
y1 = 37.29166666666667
y2 = 34.04166666666667
x1 = 140.9375
x2 = 137.0625

#Output file open
with open(output_csvfile, 'a', encoding='utf-8', newline="") as f:

    #Get the path of all csv files in the folder
    All_Files = glob.glob('./data/*.csv')
    pprint.pprint (All_Files)

    #Loop processing for the number of csv files
    for file in All_Files:
        #Store data in list
        df = pd.read_csv(file, header=None, encoding='utf-8')
        print(u'End of storage process in data frame')
        
        #Give a column name
        df.columns = ["time_s_utc","time_f_utc","sign","surface","lon","lat","rain"]
        
        print("---Before data extraction---")

        #Show the first 5 values
        print(df.head(5))

        #See dtype for each column
        print(df.dtypes)

        #Take the number of rows and columns
        print(df.shape)
        
        #Extract data in the target range
        df_kt = df[(df["lon"]>x2) & (df["lon"]<x1) & (df["lat"]>y2) & (df["lat"]<y1)]

        #Added 5th mesh code
        df_kt['meshcode'] = ju.to_meshcode(df_kt.lat, df_kt.lon, 5)

        print("---After data extraction---")

        #Show the first 5 values
        print(df_kt.head(5))

        #See dtype for each column
        print(df_kt.dtypes)

        #Take the number of rows and columns
        print(df_kt.shape)

        #Write to output CSV file
        df_kt.to_csv(f, index=False)

    print(u'Processing Exit')

Diagram of output results

Output csv file (5-minute precipitation for each 5th mesh) and 5th mesh shapefile Based on it, plot it in QGIS.

EiUsTC_VkAAhpwr.jpg

Recommended Posts

[Note] Based on the latitude and longitude of the CSV file, we created a script that extracts data in the target range and adds a mesh code.
[Python] About creating a tool to create a new Outlook email based on the data of the JSON file and the part that got caught
A Python script that reads a SQL file, executes BigQuery and saves the csv
A script that opens the URLs written in CSV in order and takes a full screen screenshot
[Note] A shell script that checks the CPU usage of a specific process in a while loop.
A note on the default behavior of collate_fn in PyTorch
[Note] Import of a file in the parent directory in Python
Process the contents of the file in order with a shell script
A python script that deletes ._DS_Store and ._ * files created on Mac
The one that divides the csv file, reads it, and processes it in parallel
Code reading of faker, a library that generates test data in Python
I created a script to check if English is entered in the specified position of the JSON file in Python.
I made a program in Python that reads CSV data of FX and creates a large amount of chart images