PTR Changelog 2018-01-18: isolated VM


  • Dev Team

    @postcrafter

    It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

    It should not be related. Are you sure it's not just coincidence?

    I already found a first bug, Game.shard seems to be undefined.

    Fixed, thanks.



  • So since we have now our own environment that means in theory parsing "the memory" is now obsolete and we are able to keep objects in it? Or is there still a chance that the entire environment gets a reset?


  • Culture

    @mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.



  • Hm makes sense. But it would allow to keep map data around for much longer without the need to decode it every time from storage so yay 🙂



  • PTR seems to be extremely slow to reload code? Something in my code was causing timeout errors, so I cutting out bigger and bigger chunks of code, sometimes it was running, most of the time I had timeouts and very often ticks would simply stop for extended time.

    Script execution timed out ungracefully, restarting virtual machine
    

    Now I'm down to this and there are no ticks at all:

    /* eslint-disable */
    'use strict';
    
    Object.defineProperty(exports, '__esModule', { value: true });
    
    function loop() {
        console.log("running");
    }
    
    exports.loop = loop;
    //# sourceMappingURL=main.js.map
    


  • that's great news !

    I'm sad to report that I'm unable to use it on PTR. With an empty main loop, no other files and empty Memory, it's still returning [3:56:35 AM][shard0]Script execution timed out ungracefully, restarting virtual machine.


  • Dev Team

    There is a critical error in isolated-vm integration, ticks are stopped currently. We're working on it.


  • Dev Team

    Server is restarted. There is a memory leak somewhere, we're trying to track it down, but it may break a few times again.



  • Ok, got it running now.

    minor nitpick, in the UI, the top right CPU/Memory might have some rounding issue. I'm seeing values like these: 60.799999999999955 / 290



  • getting this one occasionally:

    [10:25:09 PM][shard0]RangeError: Array buffer allocation failed at new ArrayBuffer (<anonymous>) at typedArrayConstructByLength (<anonymous>) at new Float64Array (native) at new Heap (__runtime__:33218:27) at Object.findRoute (__runtime__:33024:13) at Object.findExit (__runtime__:33114:30) at Object.findExitTo (__runtime__:14667:29) at Object.findPathTo (__runtime__:14831:32) at Object.moveTo (__runtime__:36365:29)



  • @ags131 said in PTR Changelog 2018-01-18: isolated VM:

    @mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

    I think it would be nice if the whole memory was automatically cached to global in this case. You'd pay for the stringify at the end of the tick every time, but the parse should only be on a reset. If there's one global instance and this "just works" I think it should be in the backend rather than user-implemented.


  • Dev Team

    @tigga This is a nice idea, we'll look into what we can do.


  • YP

    @tigga you can already do that:

    let memory: any;
    let lastTime: number = 0;
    
    export function load_memory() {
    
      // const start = Game.cpu.getUsed();
      if (lastTime && memory && Game.time === (lastTime + 1)) {
        delete global.Memory;
        global.Memory = memory;
        RawMemory._parsed = memory;
        // console.log("Tick has same GID!");
      } else {
        // noinspection TsLint
        Memory.rooms;
        memory = RawMemory._parsed;
      }
      lastTime = Game.time;
      // const end = Game.cpu.getUsed();
      // console.log("memory loaded", lastTime, end - start);
    }
    

    this snippet is from slack, not sure who posted it ,) this also will work on the live server, when 2 ticks run on the same server, which happens quite often.



  • @w4rl0ck I'd seen that before and my post reminded me of it as well! I actually implemented it today (with some help from @PostCrafter). IIUC it's a bit hacky because _parsed isn't part of the spec.


  • Culture

    PTR seems to be running really slowly right now, which is making testing harder.


  • Dev Team

    Native PathFinder is now enabled in isolated-vm server nodes.



  • PathFinder seems to be fine, but there are relatively rare errors, see below:

    Spoiler:
    RangeError: Array buffer allocation failed
        at new ArrayBuffer ()
        at typedArrayConstructByLength ()
        at new Float64Array (native)
        at new Heap (:33344:27)
        // ======= ^ Screeps space
        at Object.findRoute (:33150:13)
        at Object.Creep.moveTo2 (__prototypes:1130:34)
        at Role.work (scout:179:27)
        at legacy_loop (main:1328:60)
        at LEGACY.run (main:1425:9)
        at Kernel:44:77
        // ======= ^ userspace
    

    and

    RangeError: Array buffer allocation failed
        at new ArrayBuffer ()
        at typedArrayConstructByLength ()
        at new Uint8Array (native)
        at new  (:39248:22)
        // ======= ^ Screeps space
        at staticRoomCallback (__prototypes:356:21)
        at global.civilRoomCallback (__prototypes:421:21)
        at roomCallback (__prototypes:1151:24)
        at :33018:31
        at Object.exports.search (:33030:19)
        at Object.search (:39286:38)
        // ======= ^ userspace
    

    Moreover, sometimes V8 heap overflow errors appear unexpectedly, cannot track them for now.


  • Dev Team

    @mototroller You seem to exceed your heap memory quota, and game engine is no longer able to allocate some space for its needs.



  • Will we get isolated-vm support for private servers as well?

    I had the idea of giving bot creators a tool to create a binary of their bot's isolate snapshot that can be shared without publishing the source code. Currently bot publisher are trying to not publish a lot of offensive code, because it's going to be used on the official server, which in turn is hindering the experience of newer players. If this idea were to be implemented they could hold back parts of their code just like they do now, but publish the entire bot including military.

    Actually implementing the upload and use of a binary snapshot can be taken care of by a mod, I'm just interested in whether we will get full isolated-vm support.


  • Dev Team

    @postcrafter We'll port it to private server driver once its implementation is finalized enough. isolated-vm library still doesn't have all features enabled on Windows, for example.

    Bot snapshots is an interesting idea, but could be a bit tricky, since snapshots are architecture-dependent.

    👍