hard resets/timeouts storms

[Finndibaen]

I usually have a few timeouts/hard resets per day, but today, in a single (so a 15 mins interval) notification email i received

83 Script execution timed out: CPU limit reached
35 Script execution has been terminated: CPU bucket is empty
28 [Priority 1] TypeError: Cannot read property 'message' of null (those are exceptions without message) 

This notification email is dated 9:30 AM GMT time, according to my gathered metrics it was NOT during a reset storm.

I'm pretty sure it matches the interval where i don't have any metric in my reporting, underlined in red on the image below

edit : I'm reporting this, because if you add up all the missed ticks, it's almost 12 minutes long ... hopefully i wasn't attacked during that time

[artch]

Bad things happen sometimes. Is it an ongoing problem or happened just once?

[Finndibaen]

@artch i've never seen anything like this until now indeed (hopefully), bad 15 mins of no service is pretty bad on the bad scale.

actually ... I hadn't thorougly checked the emails... FR times

11:30 AM : 
83	Script execution timed out: CPU limit reached
35	Script execution has been terminated: CPU bucket is empty
28	[Priority 1] TypeError: Cannot read property 'message' of null
11:36 AM
40	[Priority 1] TypeError: Cannot read property 'message' of null
11:42 AM
36	[Priority 1] TypeError: Cannot read property 'message' of null
11:47 AM
28	[Priority 1] TypeError: Cannot read property 'message' of null
11:53 AM
29	[Priority 1] TypeError: Cannot read property 'message' of null
12:00 PM
168	Script execution timed out: CPU limit reached
99	Script execution has been terminated: CPU bucket is empty

here is the cpu view, as you can see, my code was blacked out for almost 30 mins

[artch]

It might be related to this bug which is fixed just now.

[Finndibaen]

so indeed bucket is now refilled during storms, but there also seem to be very bad storms going on right now.

I'm currently experiencing one for almost 8H

i can say exactly when you pushed that fix to live though !

[Finndibaen]

this seems over for now; lasted about 16h though