Reset Storm bucket reset not working
-
There have been 30 resets in the last few minutes, and my bucket hasn't been boosted once.
-
We're trying to fine tune it right now. Bucket is now increased by 1,000 CPU on every restart gradually, instead of replenishing to 10,000.
-
Also, bucket is boosted only when planned restarts happen. We cannot really detect an accident restart that are due to some runtime process failures, and link it to particular users.
-
That's not enough to cover the damage that these reset storms do. This one were 62 global resets, which means you're only giving about 16 cpu back per credit reset, which doesn't cover the damage caused by it. In order to run the same amount of code each tick my bucket dropped by over 2k.
-
I know you only do this with the planned ones, and that's fine with me as the planned ones are the real problem due to their clustering together. A random reset or two is barely noticeable- several dozen though makes a big dent.
-
I think "locking" your bucket at a certain level during restarts could be valid.
Let's say you got 1000 CPU in bucket and restarts happen. It should fluctuate around 1000 during that.
Might even add a modifier for CPU during this period of 1.5 (450 used becomes 300 for example)
-
Personally I would be fine with the existing system, just with different numbers. Instead of adding 1k to the bucket add 3k (still split up) and see if that's better. Enough of us have graphing and stats tools where we should be able to provide feedback pretty quickly.
-
Could you please share your CPU graphs here with the current set up? After that, we'll change it to 3K and compare the graphs then.
-
@artem
This is my current bucket-to-cpu ratio:
http://i.imgur.com/CxoSjb1.png
Yellow line is bucket ( top is full, bottom is empty )
