Memory loading CPU usage is all over the place

Dissi

Each tick it swings between 20~50 CPU:

http://i.imgur.com/FXowv6r.png

I've optimized it to bits and it's still causing headaches. Can we get some more information on why it can vary this wildly? I'd like to avoid it at all costs.

artch

The deserialization process is as simple as this:

global.Memory = JSON.parse(RawMemory.get());

There is no more code involved. RawMemory.get() returns a string that is cached in memory already, it has no cost.

The variation in cost may be explained by random GC runs during execution, or some inner heuristic variative optimizations ofJSON.parse in V8.

Voronoi

@dissi mine is relatively steady.
Doesn't look to be a db fetch issue.
Peaks on my case are pbly GC related or process slowdown/switch server side.

Voronoi

Errata:
If I profile the memory explicitely:

var log = console.log;
var stringified = JSON.stringify(Memory);
var startCpu = Game.cpu.getUsed();

var memory = null;
var count = 10;

for (var i = 0; i < count; i++) {
memory = JSON.parse(stringified);
}
var endCpu = Game.cpu.getUsed();
log('CPU spent on Memory parsing:', + (endCpu - startCpu) / count);

I have a consistent time which is half the time of my initial parse triggered when accessing memory for the first time.
This means, 2.3 vs 4.6 on average.

@artem Could you check that you don't parse the memory twice for some reason ?
Sounds silly but ...

Dissi

After testing more, first access seems to be about 30 cpu average. When testing manually I barely touch the 15 cpu mark.
I agree with voronoi, something seems to be off.

artch

You can investigate it yourself. Here is Memory deserialization code:

Object.defineProperty(runCodeCache[userId].globals, 'Memory', {
    configurable: true,
    enumerable: true,
    get() {
    <span class="hljs-keyword">try</span> {
        runCodeCache[userId].memory._parsed = JSON.parse(runCodeCache[userId].memory.<span class="hljs-keyword">get</span>() || <span class="hljs-string">"{}"</span>);
        runCodeCache[userId].memory._parsed.__proto__ = <span class="hljs-literal">null</span>;
    }
    <span class="hljs-keyword">catch</span>(e) {
        runCodeCache[userId].memory._parsed = <span class="hljs-literal">null</span>;
    }

    <span class="hljs-built_in">Object</span>.defineProperty(runCodeCache[userId].globals, <span class="hljs-string">'Memory'</span>, {
        configurable: <span class="hljs-literal">true</span>,
        enumerable: <span class="hljs-literal">true</span>,
        value: runCodeCache[userId].memory._parsed
    });

    <span class="hljs-keyword">return</span> runCodeCache[userId].memory._parsed;
}

});

And here is runCodeCache[userId].memory object code:

var rawMemory = Object.create(null, {
    get: {
        value: function() {
            return runtimeData.userMemory.data;
        }
    },
    set: {
        value: function (value) {
            if (!_.isString(value)) {
                throw new Error('Raw memory value is not a string');
            }
            if (value.length > 2 * 1024 * 1024) {
                throw new Error('Raw memory length exceeded 2 MB limit');
            }
            if (this._parsed) {
                delete this._parsed;
            }
            runtimeData.userMemory.data = value;
        }
    }
});

Voronoi

Thanks Artem, I'll look into this further.
The extra time on the first loads could be due to V8 hidden classes compilation.
I'll try to roll out a consistent test, if this is the case, there is nothing you could do about it I guess.
But WE could do something about it, consistency pay off in javascript