Currently we have the ability to schedule jobs every 1, 2, 3, 4, 5, 15, 30 minutes.
If a have alot of spiders running on every 15 minutes, they would all try to run at the exact same time and block each other since there's a limit to how many Units i have.
One way to work around this is do create 4 jobs with the exact minutes of the hour I want to run the spider on as to spread them out over the hour. However, it's not a very neat solutions since it requires 4 jobs instead of 1 per spider.
I would like to have to ability to in the scheduling tool also select an offset in minutes. So if I selected every 15 minutes and offset 3 minutes, it would run at :03, :18, :33 and :48.
Currently we have the ability to schedule jobs every 1, 2, 3, 4, 5, 15, 30 minutes.
If a have alot of spiders running on every 15 minutes, they would all try to run at the exact same time and block each other since there's a limit to how many Units i have.
One way to work around this is do create 4 jobs with the exact minutes of the hour I want to run the spider on as to spread them out over the hour. However, it's not a very neat solutions since it requires 4 jobs instead of 1 per spider.
I would like to have to ability to in the scheduling tool also select an offset in minutes. So if I selected every 15 minutes and offset 3 minutes, it would run at :03, :18, :33 and :48.
0 Votes
0 Comments
Login to post a comment