Crawl practice with Beautiful Soup

It is like this

-Python2.7

-*- encoding: utf-8 -*-

import logging, sys, urllib
import bs4

url = 'http://www.bus-sagasu.com/searches/{}-{}/{}-{}-{}/sort:1/'.format(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5])
soup = bs4.BeautifulSoup(urllib.urlopen(url).read(),"lxml")

for result in soup.select("div.resultContent"):
  for title in result.select("div.title"):
    print title.text.strip(),
  for seat in result.select("td.seats"):
    print seat.text,
  for price in result.select("td.price"):
    print price.text,
  print ""

Recommended Posts

Crawl practice with Beautiful Soup
Scraping with Beautiful Soup
Table scraping with Beautiful Soup
Beautiful Soup
Try scraping with Python + Beautiful Soup
Scraping multiple pages with Beautiful Soup
Scraping pages with pagination with Beautiful Soup
Scraping with Beautiful Soup in 10 minutes
Website scraping with Python's Beautiful Soup
Beautiful Soup memo
Beautiful soup spills
My Beautiful Soup (Python)
Note that I dealt with HTML in Beautiful Soup
A memorandum when using beautiful soup
[Python] A memorandum of beautiful soup4
Draw a beautiful circle with numpy
Settings when using Python 3 requests and Beautiful Soup with crostini on Chromebook
Sort anime faces by scraping anime character pages with Beautiful Soup and Selenium