I have an spider with a Lua script for click a link and wait ... but, when i run my spider using crawlera ... I got a response with the initial html without the changes generated by the Lua script.
This is my Lua Script
assert(splash:go(args.url))
assert(splash:wait(1))
while not splash:select('.TablaForm3') do
splash:wait(0.1)
end
return {
html = splash:html()
}
And my spider star request method
url,
self.parse,
meta={'nog':nog, 'step': i},
headers=headers,
endpoint="execute",
args={ 'wait': 0.5, 'lua_source': scripts[i] },
dont_filter=True)
Joel Ibaceta
I have an spider with a Lua script for click a link and wait ... but, when i run my spider using crawlera ... I got a response with the initial html without the changes generated by the Lua script.
This is my Lua Script
assert(splash:go(args.url))
assert(splash:wait(1))
while not splash:select('.TablaForm3') do
splash:wait(0.1)
end
return {
html = splash:html()
}
end
And my spider star request method
url,
self.parse,
meta={'nog':nog, 'step': i},
headers=headers,
endpoint="execute",
args={ 'wait': 0.5, 'lua_source': scripts[i] },
dont_filter=True)