PTR Changelog 2018-01-18: isolated VM

artch

This post describes changes on the Public Test Realm. The estimated launch date is March 5.

We're happy to introduce a new major feature that may impact your Screeps gameplay in a whole new way: isolated virtual machine. We explained this a bit in this blog post earlier.

TL;DR:

You can opt-in for an isolated VM in the new account settings UI called Runtime. This will move execution of your script to another pool of server nodes with this experimental feature enabled.
In isolated VM you have your very own JavaScript environment, including heap memory and garbage collector.
Since isolated VMs can be transferred between threads within the same Node.js process, it is no longer neccessary to maintain multiple global instances, you always have one single global.
You can use new API method Game.cpu.getHeapStatistics() to know how much heap memory you're using in runtime. Maximum effective limit is 256 MB for each player.

Please note that this feature is still in development and considered experimental. Please report any incorrect behavior and be ready that things will be improved over time.

PostCrafter

Great news, nice to see this implemented.

It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

I already found a first bug, Game.shard seems to be undefined.

Hiryus

I confirm both remarks from @postcrafter.

artch

@postcrafter

It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

It should not be related. Are you sure it's not just coincidence?

I already found a first bug, Game.shard seems to be undefined.

Fixed, thanks.

MrFaul

So since we have now our own environment that means in theory parsing "the memory" is now obsolete and we are able to keep objects in it? Or is there still a chance that the entire environment gets a reset?

ags131

@mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

MrFaul

Hm makes sense. But it would allow to keep map data around for much longer without the need to decode it every time from storage so yay

UnFleshedOne

PTR seems to be extremely slow to reload code? Something in my code was causing timeout errors, so I cutting out bigger and bigger chunks of code, sometimes it was running, most of the time I had timeouts and very often ticks would simply stop for extended time.

Script execution timed out ungracefully, restarting virtual machine

Now I'm down to this and there are no ticks at all:

/* eslint-disable */
'use strict';

Object.defineProperty(exports, '__esModule', { value: true });

function loop() {
    console.log("running");
}

exports.loop = loop;
//# sourceMappingURL=main.js.map

vrs

that's great news !

I'm sad to report that I'm unable to use it on PTR. With an empty main loop, no other files and empty Memory, it's still returning [3:56:35 AM][shard0]Script execution timed out ungracefully, restarting virtual machine.

artch

There is a critical error in isolated-vm integration, ticks are stopped currently. We're working on it.

artch

Server is restarted. There is a memory leak somewhere, we're trying to track it down, but it may break a few times again.

vrs

Ok, got it running now.

minor nitpick, in the UI, the top right CPU/Memory might have some rounding issue. I'm seeing values like these: 60.799999999999955 / 290

vrs

getting this one occasionally:

[10:25:09 PM][shard0]RangeError: Array buffer allocation failed at new ArrayBuffer (<anonymous>) at typedArrayConstructByLength (<anonymous>) at new Float64Array (native) at new Heap (__runtime__:33218:27) at Object.findRoute (__runtime__:33024:13) at Object.findExit (__runtime__:33114:30) at Object.findExitTo (__runtime__:14667:29) at Object.findPathTo (__runtime__:14831:32) at Object.moveTo (__runtime__:36365:29)

Tigga

@ags131 said in PTR Changelog 2018-01-18: isolated VM:

@mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

I think it would be nice if the whole memory was automatically cached to global in this case. You'd pay for the stringify at the end of the tick every time, but the parse should only be on a reset. If there's one global instance and this "just works" I think it should be in the backend rather than user-implemented.

artch

@tigga This is a nice idea, we'll look into what we can do.

W4rl0ck

@tigga you can already do that:

let memory: any;
let lastTime: number = 0;

export function load_memory() {

  // const start = Game.cpu.getUsed();
  if (lastTime && memory && Game.time === (lastTime + 1)) {
    delete global.Memory;
    global.Memory = memory;
    RawMemory._parsed = memory;
    // console.log("Tick has same GID!");
  } else {
    // noinspection TsLint
    Memory.rooms;
    memory = RawMemory._parsed;
  }
  lastTime = Game.time;
  // const end = Game.cpu.getUsed();
  // console.log("memory loaded", lastTime, end - start);
}

this snippet is from slack, not sure who posted it ,) this also will work on the live server, when 2 ticks run on the same server, which happens quite often.

Tigga

@w4rl0ck I'd seen that before and my post reminded me of it as well! I actually implemented it today (with some help from @PostCrafter). IIUC it's a bit hacky because _parsed isn't part of the spec.

tedivm

PTR seems to be running really slowly right now, which is making testing harder.

artch

Native PathFinder is now enabled in isolated-vm server nodes.

Mototroller

PathFinder seems to be fine, but there are relatively rare errors, see below:

Spoiler:

RangeError: Array buffer allocation failed
    at new ArrayBuffer ()
    at typedArrayConstructByLength ()
    at new Float64Array (native)
    at new Heap (:33344:27)
    // ======= ^ Screeps space
    at Object.findRoute (:33150:13)
    at Object.Creep.moveTo2 (__prototypes:1130:34)
    at Role.work (scout:179:27)
    at legacy_loop (main:1328:60)
    at LEGACY.run (main:1425:9)
    at Kernel:44:77
    // ======= ^ userspace

and

RangeError: Array buffer allocation failed
    at new ArrayBuffer ()
    at typedArrayConstructByLength ()
    at new Uint8Array (native)
    at new  (:39248:22)
    // ======= ^ Screeps space
    at staticRoomCallback (__prototypes:356:21)
    at global.civilRoomCallback (__prototypes:421:21)
    at roomCallback (__prototypes:1151:24)
    at :33018:31
    at Object.exports.search (:33030:19)
    at Object.search (:39286:38)
    // ======= ^ userspace

Moreover, sometimes V8 heap overflow errors appear unexpectedly, cannot track them for now.

artch

@mototroller You seem to exceed your heap memory quota, and game engine is no longer able to allocate some space for its needs.