[Treasure Data] [Python] Execute a query on Treasure Data using TD Client

background

On the GUI Treasure Workflow

+run_query:
  td_run>: sample_query

You can execute the query with the query name on Treasure Data. However, it seems that there is no such function on the Python Client side. (Might happen)

Conclusion

** Use run_schedule **

import time
import tdclient

if __name__ == '__main__':
    query_name = 'sample_query'
    with tdclient.Client(apikey='hogehoge') as td:
        # get unix time
        unix_time = int(time.time())
        run_time = unix_time + 10

        # set schedule query
        # run now!
        res = td.run_schedule(name=query_name, time=run_time, num=1)
    return 0

Bonus-I want to wait for execution / I want results

import time
import tdclient

def main():
    query_name = 'sample_query'
    with tdclient.Client(apikey='hogehoge') as td:
        unix_time = int(time.time())
        run_time = unix_time + 10

        # set schedule query
        # run now!
        res = td.run_schedule(name=query_name, time=run_time, num=1)
        schedule_job = res[0]

        # get job_id from ScheduleJob object
        job_id = schedule_job._job_id

        # get Job object by job_id
        job = td.job(job_id=job_id)

        # wait until job finished
        job.wait()

        # get results one by one
        for row in job.result():
            print(repr(row))
    return 0


if __name__ == '__main__':
    main()

Postscript

The reason why I wanted to hit the query on Treasure Data is ... I wanted to use the one set with Connector on the GUI of Treasure Data. Of course, I could set Export on Python, but I wanted to wear it sideways.

reference

Official-Python Client GitHub - Treasure Data API library for Python

Recommended Posts

[Treasure Data] [Python] Execute a query on Treasure Data using TD Client
Execute Python code on C ++ (using Boost.Python)
Periodically execute Python Script on AWS Data Pipeline
Folium: Visualize data on a map with Python
Data analysis using Python 0
Data cleaning using Python
[Grasshopper] When creating a data tree on Python script
Create a data collection bot in Python using Selenium
Dry-run sql query using psycopg2 on Redshift in Python
Build a Python environment on your Mac using pyenv
Build a Python development environment using pyenv on MacOS
How to execute a command using subprocess in Python
Try using Kubernetes Client -Python-
Building a Python environment on a Mac and using Jupyter lab
[Python] Chapter 01-03 About Python (Write and execute a program using PyCharm)
Detect "temperature (using A / D converter)" using python on Raspberry Pi 3!
[Python] Notes on data analysis
Build a python data analysis environment on Mac (El Capitan)
Broadcast on LINE using python
Data analysis using python pandas
Until building a Python development environment using pyenv on Ubuntu 20.04
Play with YouTube Data API v3 using Google API Python Client
Control the motor with a motor driver using python on Raspberry Pi 3!
[Python] View data on new coronavirus infections on a 7-day moving average
Get data from MySQL on a VPS with Python 3 and SQLAlchemy
How to build a Python environment using Virtualenv on Ubuntu 18.04 LTS
[Python] Analyze Splatoon 2 league match data using a correlation coefficient table
How to update a Tableau packaged workbook data source using Python
A note on using tab completion when running Python interactively on Windows
[Python] Read a csv file with a large data size using a generator
Get data from your website on a regular basis using ScraperWiki
Building a Python environment on Mac
Data acquisition using python googlemap api
I made a Line-bot using Python!
Hit treasure data from Python Pandas
Create a python GUI using tkinter
Building a Python environment on Ubuntu
Introducing Python using pyenv on Ubuntu 20.04
Drawing a silverstone curve using python
Notes on using MeCab from Python
Preparing python using vscode on ubuntu
Create a Python environment on Mac (2017/4)
Study on Tokyo Rent Using Python (3-2)
Notes on installing Python using PyEnv
Create a python environment on centos
Using a serial console on Ubuntu 20.04
Notes on using rstrip with python.
Install Python on CentOS using Pyenv
Study on Tokyo Rent Using Python (3-3)
Build a python3 environment on CentOS7
Install Python on CentOS using pyenv
A simple HTTP client implemented in Python
Detect "brightness" using python on Raspberry Pi 3!
Add a Python data source with Redash
Build a python environment on MacOS (Catallina)
Create a python environment on your Mac
Install python library on Lambda using [/ tmp]
A memo with Python2.7 and Python3 on CentOS
Map rent information on a map with python
Creating a python virtual environment on Windows
[Python] Various data processing using Numpy arrays