PTR Changelog 2018-01-18: isolated VM

artch

It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

It should not be related. Are you sure it's not just coincidence?

I already found a first bug, Game.shard seems to be undefined.

Fixed, thanks.

MrFaul

So since we have now our own environment that means in theory parsing "the memory" is now obsolete and we are able to keep objects in it? Or is there still a chance that the entire environment gets a reset?

ags131

@mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

MrFaul

Hm makes sense. But it would allow to keep map data around for much longer without the need to decode it every time from storage so yay

UnFleshedOne

PTR seems to be extremely slow to reload code? Something in my code was causing timeout errors, so I cutting out bigger and bigger chunks of code, sometimes it was running, most of the time I had timeouts and very often ticks would simply stop for extended time.

Script execution timed out ungracefully, restarting virtual machine

Now I'm down to this and there are no ticks at all:

/* eslint-disable */
'use strict';

Object.defineProperty(exports, '__esModule', { value: true });

function loop() {
    console.log("running");
}

exports.loop = loop;
//# sourceMappingURL=main.js.map

vrs

that's great news !

I'm sad to report that I'm unable to use it on PTR. With an empty main loop, no other files and empty Memory, it's still returning [3:56:35 AM][shard0]Script execution timed out ungracefully, restarting virtual machine.

artch

There is a critical error in isolated-vm integration, ticks are stopped currently. We're working on it.

artch

Server is restarted. There is a memory leak somewhere, we're trying to track it down, but it may break a few times again.

vrs

Ok, got it running now.

minor nitpick, in the UI, the top right CPU/Memory might have some rounding issue. I'm seeing values like these: 60.799999999999955 / 290

vrs

getting this one occasionally:

[10:25:09 PM][shard0]RangeError: Array buffer allocation failed at new ArrayBuffer (<anonymous>) at typedArrayConstructByLength (<anonymous>) at new Float64Array (native) at new Heap (__runtime__:33218:27) at Object.findRoute (__runtime__:33024:13) at Object.findExit (__runtime__:33114:30) at Object.findExitTo (__runtime__:14667:29) at Object.findPathTo (__runtime__:14831:32) at Object.moveTo (__runtime__:36365:29)

Tigga

@ags131 said in PTR Changelog 2018-01-18: isolated VM:

@mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

I think it would be nice if the whole memory was automatically cached to global in this case. You'd pay for the stringify at the end of the tick every time, but the parse should only be on a reset. If there's one global instance and this "just works" I think it should be in the backend rather than user-implemented.

artch

@tigga This is a nice idea, we'll look into what we can do.

W4rl0ck

@tigga you can already do that:

let memory: any;
let lastTime: number = 0;

export function load_memory() {

  // const start = Game.cpu.getUsed();
  if (lastTime && memory && Game.time === (lastTime + 1)) {
    delete global.Memory;
    global.Memory = memory;
    RawMemory._parsed = memory;
    // console.log("Tick has same GID!");
  } else {
    // noinspection TsLint
    Memory.rooms;
    memory = RawMemory._parsed;
  }
  lastTime = Game.time;
  // const end = Game.cpu.getUsed();
  // console.log("memory loaded", lastTime, end - start);
}

this snippet is from slack, not sure who posted it ,) this also will work on the live server, when 2 ticks run on the same server, which happens quite often.

Tigga

@w4rl0ck I'd seen that before and my post reminded me of it as well! I actually implemented it today (with some help from @PostCrafter). IIUC it's a bit hacky because _parsed isn't part of the spec.

tedivm

PTR seems to be running really slowly right now, which is making testing harder.

artch

Native PathFinder is now enabled in isolated-vm server nodes.

Mototroller

PathFinder seems to be fine, but there are relatively rare errors, see below:

Spoiler:

RangeError: Array buffer allocation failed
    at new ArrayBuffer ()
    at typedArrayConstructByLength ()
    at new Float64Array (native)
    at new Heap (:33344:27)
    // ======= ^ Screeps space
    at Object.findRoute (:33150:13)
    at Object.Creep.moveTo2 (__prototypes:1130:34)
    at Role.work (scout:179:27)
    at legacy_loop (main:1328:60)
    at LEGACY.run (main:1425:9)
    at Kernel:44:77
    // ======= ^ userspace

and

RangeError: Array buffer allocation failed
    at new ArrayBuffer ()
    at typedArrayConstructByLength ()
    at new Uint8Array (native)
    at new  (:39248:22)
    // ======= ^ Screeps space
    at staticRoomCallback (__prototypes:356:21)
    at global.civilRoomCallback (__prototypes:421:21)
    at roomCallback (__prototypes:1151:24)
    at :33018:31
    at Object.exports.search (:33030:19)
    at Object.search (:39286:38)
    // ======= ^ userspace

Moreover, sometimes V8 heap overflow errors appear unexpectedly, cannot track them for now.

artch

@mototroller You seem to exceed your heap memory quota, and game engine is no longer able to allocate some space for its needs.

PostCrafter

Will we get isolated-vm support for private servers as well?

I had the idea of giving bot creators a tool to create a binary of their bot's isolate snapshot that can be shared without publishing the source code. Currently bot publisher are trying to not publish a lot of offensive code, because it's going to be used on the official server, which in turn is hindering the experience of newer players. If this idea were to be implemented they could hold back parts of their code just like they do now, but publish the entire bot including military.

Actually implementing the upload and use of a binary snapshot can be taken care of by a mod, I'm just interested in whether we will get full isolated-vm support.

artch

@postcrafter We'll port it to private server driver once its implementation is finalized enough. isolated-vm library still doesn't have all features enabled on Windows, for example.

Bot snapshots is an interesting idea, but could be a bit tricky, since snapshots are architecture-dependent.