Cette fois, j'ai résumé les solutions pour le phénomène de rollback lors de l'exécution sur la console. Cette fois, je vais vous expliquer en utilisant le grattage comme exemple. Il y a un commentaire dans la seconde moitié.
scraping.rb
class Scraping < ApplicationRecord
def self.get_infomation
require 'mechanize'
agent = Mechanize.new
links = []
current_page = agent.get("https://talent-dictionary.com/s/jobs/3/20")
elements = current_page.at('.home_talent_list_wrapper')
boxs = elements.search('.item')
roks = boxs.search('.right')
qqqs = roks.search('a')
eees = qqqs.search('.title')
eees.each do |eee|
links << eee.inner_text
end
links.each do |link|
get_personal_infomation('https://talent-dictionary.com/' + link)
end
end
def self.get_personal_infomation(link)
agent = Mechanize.new
personal_page = agent.get(link)
aaas = personal_page.at('.talent_name_wrapper')
ages = aaas.at('.age').inner_text.delete('âge').to_i if aaas.at('.age')
names = aaas.at('h1').inner_text if aaas.at('h1')
image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
infomation = Infomation.where(name: names).first_or_initialize
infomation.age = ages
infomation.image_url = image_urls
infomation.save
byebug
end
end
Exécutez get_infomation dans la console.
[1] pry(main)> Scraping.get_infomation
(0.3ms) SET NAMES utf8, @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
Infomation Load (0.2ms) SELECT `infomations`.* FROM `infomations` WHERE `infomations`.`name` = 'Étain Hirose' ORDER BY `infomations`.`id` ASC LIMIT 1
(0.1ms) BEGIN
(0.1ms) ROLLBACK
Return value is: nil
[23, 32] in /home/ec2-user/environment/filebook/app/models/scraping.rb
23: ages = aaas.at('.age').inner_text.delete('âge').to_i if aaas.at('.age')
24: names = aaas.at('h1').inner_text if aaas.at('h1')
25: image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
26: infomation = Infomation.where(name: names).first_or_initialize
27: infomation.age = ages
28: infomation.image_url = image_urls
29: infomation.save
30: byebug
=> 31: end
32: end
(byebug)
En regardant ce résultat, je ne comprends pas pourquoi la restauration est effectuée. Il y a une commande qui va cracher la raison de cette restauration! !! !!
Il y a deux étapes. La première rue est
aplication.rb
require_relative 'boot'
require 'rails/all'
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)
module Filebook
class Application < Rails::Application
# Initialize configuration defaults for originally generated Rails version.
config.load_defaults 5.2
config.logger = Logger.new(STDOUT)← Description de cette colonne
# Settings in config/environments/* take precedence over those specified here.
# Application configuration can go into files in config/initializers
# -- all .rb files in that directory are automatically loaded after loading
# the framework and any gems in your application.
end
end
2ème étape
scraping.rb
class Scraping < ApplicationRecord
def self.get_infomation
require 'mechanize'
agent = Mechanize.new
links = []
current_page = agent.get("https://talent-dictionary.com/s/jobs/3/20")
elements = current_page.at('.home_talent_list_wrapper')
boxs = elements.search('.item')
roks = boxs.search('.right')
qqqs = roks.search('a')
eees = qqqs.search('.title')
eees.each do |eee|
links << eee.inner_text
end
links.each do |link|
get_personal_infomation('https://talent-dictionary.com/' + link)
end
end
def self.get_personal_infomation(link)
agent = Mechanize.new
personal_page = agent.get(link)
aaas = personal_page.at('.talent_name_wrapper')
ages = aaas.at('.age').inner_text.delete('âge').to_i if aaas.at('.age')
names = aaas.at('h1').inner_text if aaas.at('h1')
image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
infomation = Infomation.where(name: names).first_or_initialize
infomation.age = ages
infomation.image_url = image_urls
infomation.save
logger.debug infomation.errors.inspecter ← Cette description
byebug
end
end
Si vous mettez ces deux étapes entre
[1] pry(main)> Scraping.get_infomation
D, [2020-11-02T13:39:29.459445 #5812] DEBUG -- : (0.3ms) SET NAMES utf8, @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
D, [2020-11-02T13:39:29.464122 #5812] DEBUG -- : Infomation Load (0.2ms) SELECT `infomations`.* FROM `infomations` WHERE `infomations`.`name` = 'Étain Hirose' ORDER BY `infomations`.`id` ASC LIMIT 1
D, [2020-11-02T13:39:29.470351 #5812] DEBUG -- : (0.1ms) BEGIN
D, [2020-11-02T13:39:29.476168 #5812] DEBUG -- : (0.1ms) ROLLBACK
D, [2020-11-02T13:39:29.476801 #5812] DEBUG -- : #<ActiveModel::Errors:0x0000000004383ef8 @base=#<Infomation id: 14, age: 22, name: "Étain Hirose", image_url: "https://images.talent-dictionary.com/uploads/image...", created_at: "2020-11-01 07:14:44", updated_at: "2020-11-01 07:14:44">, @messages={:user=>["must exist"]}, @details={:user=>[{:error=>:blank}]}>
Return value is:nil ↑ Description ci-dessus
[24, 33] in /home/ec2-user/environment/filebook/app/models/scraping.rb
24: names = aaas.at('h1').inner_text if aaas.at('h1')
25: image_urls = personal_page.at('.main_image img').get_attribute('src') if personal_page.at('.main_image img')
26: infomation = Infomation.where(name: names).first_or_initialize
27: infomation.age = ages
28: infomation.image_url = image_urls
29: infomation.save
30: logger.debug infomation.errors.inspect
31: byebug
=> 32: end
33: end
(byebug)
<ActiveModel::Errors:0x0000000004383ef8 @base=#<Infomation id: 14, age: 22, name: "Étain Hirose", image_url: "https://images.talent-dictionary.com/uploads/image...", created_at: "2020-11-01 07:14:44", updated_at: "2020-11-01 07:14:44">, @messages={:user=>["must exist"]}, @details={:user=>[{:error=>:blank}]}>
Je crache une erreur comme celle-ci.
Cette erreur se traduit par {: user => ["must exist"] l'utilisateur est requis, mais {: user => [{: error =>: blank}] Mais il peut être traduit que la colonne utilisateur est vide.
Semble être une erreur d'association.
Seconde moitié
Un enregistreur crache des journaux (indiquant ce qui a été fait) des contrôleurs, des modèles, des vues, etc. La fonctionnalité de logger se compose de la classe Logger, qui hérite de la classe ActiveSupport.
・ Config / application.rb ・ Config / evironments / xxxxxxx.rb -Dans un fichier qui a une classe qui hérite d'ActiveSupport Tu peux l'utiliser.
Vous pouvez spécifier le fichier pour enregistrer le journal en écrivant Logger.new ('nom de fichier'). (STDOUT) est enregistré dans le fichier 'log / development.log' par défaut
Ceci est une étape. Dans un deuxième temps, nous écrirons le code dans le fichier à traiter. logger.debug xxxxx.errors.inspect Si vous écrivez errors.inspect, il inspectera les erreurs.
Veuillez l'essayer.
Recommended Posts