Are you using shub deploy from the project folder?
L
Length Power
said
almost 7 years ago
Answer
Using shub deploy from the project folder ,solved.
V
Vincent Randal
said
almost 6 years ago
I'm glad that worked, but please clear about the details. What is "the project" and where is it? I'm following the example on https://scrapy.org/ and I get this error after attempting to run shub deploy
Error: Cannot find project: There is no scrapinghub.yml, scrapy.cfg, or Dockerfile in this directory or any of the parent directories.
Length Power
Here is my first spider created on local pc.
scrapy startproject project
cd project
scrapy genspider quotes quotes.toscrape.com
vim project/items.py
import scrapy
class ProjectItem(scrapy.Item):
quote = scrapy.Field()
author = scrapy.Field()
vim project/spiders/quotes.py
# -*- coding: utf-8 -*-
import scrapy
from project.items import ProjectItem
class QuotesSpider(scrapy.Spider):
name = 'quotes'
allowed_domains = ['quotes.toscrape.com']
start_urls = ['http://quotes.toscrape.com/']
def parse(self, response):
item = ProjectItem()
for quote in response.css('div.quote'):
item['quote'] = quote.css('span.text::text').extract_first()
item['author'] = quote.xpath('span/small/text()').extract_first()
yield item
next_page = response.css('li.next a::attr("href")').extract_first()
if next_page is not None:
yield response.follow(next_page, self.parse)
I can scrawl many quotes with command
scrapy crawl quotes -o /tmp/quotes.json -t json
Now i want to deploy it into scrapinghub.
In my local pc,
sudo pip install shub --upgrade
shub deploy
Error: Cannot find project: There is no scrapinghub.yml, scrapy.cfg, or Dockerfile in this directory or any of the parent directories.
How to deploy my spider into scrapinghub ?
Using shub deploy from the project folder ,solved.
- Oldest First
- Popular
- Newest First
Sorted by Oldest Firstnestor
Are you using shub deploy from the project folder?
Length Power
Using shub deploy from the project folder ,solved.
Vincent Randal
I'm glad that worked, but please clear about the details. What is "the project" and where is it? I'm following the example on https://scrapy.org/ and I get this error after attempting to run shub deploy
Error: Cannot find project: There is no scrapinghub.yml, scrapy.cfg, or Dockerfile in this directory or any of the parent directories.
-
Unable to select Scrapy project in GitHub
-
ScrapyCloud can't call spider?
-
Unhandled error in Deferred
-
Item API - Filtering
-
newbie to web scraping but need data from zillow
-
ValueError: Invalid control character
-
Cancelling account
-
Best Practices
-
Beautifulsoup with ScrapingHub
-
Delete a project in ScrapingHub
See all 460 topics