Create rate limiting behavior #18

I need a way to create rate limiting behavior for downloads.

My initial thoughts is to make it only machine specific, and have a semaphore or something similar where a 503 or similar error leads to an increase in "wait" times. Maybe have a random delay per thread added to space out requests?

I need a way to create rate limiting behavior for downloads. My initial thoughts is to make it only machine specific, and have a semaphore or something similar where a 503 or similar error leads to an increase in "wait" times. Maybe have a random delay per thread added to space out requests?

waiting has two parts

reset
delay

If something cancels, it flags a reset wait for all processes. They pause and wait for the reset, and then begin. maybe add a rv to the wait.

It also increments a counter, and log10(counter) is the number of seconds to wait between requests

waiting has two parts 1. reset 1. delay If something cancels, it flags a reset wait for all processes. They pause and wait for the reset, and then begin. maybe add a rv to the wait. It also increments a counter, and log10(counter) is the number of seconds to wait between requests

Done, committed in ff2c5b9. Still needs tested, but it worked fine as far as I can tell.

Done, committed in ff2c5b9. Still needs tested, but it worked fine as far as I can tell.

Labels Milestones

Create rate limiting behavior #18