PTR Changelog 2018-01-18: isolated VM


  • Dev Team

    This post describes changes on the Public Test Realm. The estimated launch date is March 5.

    We're happy to introduce a new major feature that may impact your Screeps gameplay in a whole new way: isolated virtual machine. We explained this a bit in this blog post earlier.

    TL;DR:

    • You can opt-in for an isolated VM in the new account settings UI called Runtime. This will move execution of your script to another pool of server nodes with this experimental feature enabled. 0_1516270538209_chrome_2018-01-18_13-15-02.png

    • In isolated VM you have your very own JavaScript environment, including heap memory and garbage collector.

    • Since isolated VMs can be transferred between threads within the same Node.js process, it is no longer neccessary to maintain multiple global instances, you always have one single global.

    • You can use new API method Game.cpu.getHeapStatistics() to know how much heap memory you're using in runtime. Maximum effective limit is 256 MB for each player.

    Please note that this feature is still in development and considered experimental. Please report any incorrect behavior and be ready that things will be improved over time.



  • Great news, nice to see this implemented.

    It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

    I already found a first bug, Game.shard seems to be undefined.



  • I confirm both remarks from @postcrafter.


  • Dev Team

    @postcrafter

    It seems like you currently need to save your code after switching to isolated mode, in order to force a code reload to the new virtual machine.

    It should not be related. Are you sure it's not just coincidence?

    I already found a first bug, Game.shard seems to be undefined.

    Fixed, thanks.



  • So since we have now our own environment that means in theory parsing "the memory" is now obsolete and we are able to keep objects in it? Or is there still a chance that the entire environment gets a reset?


  • Culture

    @mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.



  • Hm makes sense. But it would allow to keep map data around for much longer without the need to decode it every time from storage so yay 🙂



  • PTR seems to be extremely slow to reload code? Something in my code was causing timeout errors, so I cutting out bigger and bigger chunks of code, sometimes it was running, most of the time I had timeouts and very often ticks would simply stop for extended time.

    Script execution timed out ungracefully, restarting virtual machine
    

    Now I'm down to this and there are no ticks at all:

    /* eslint-disable */
    'use strict';
    
    Object.defineProperty(exports, '__esModule', { value: true });
    
    function loop() {
        console.log("running");
    }
    
    exports.loop = loop;
    //# sourceMappingURL=main.js.map
    


  • that's great news !

    I'm sad to report that I'm unable to use it on PTR. With an empty main loop, no other files and empty Memory, it's still returning [3:56:35 AM][shard0]Script execution timed out ungracefully, restarting virtual machine.


  • Dev Team

    There is a critical error in isolated-vm integration, ticks are stopped currently. We're working on it.


  • Dev Team

    Server is restarted. There is a memory leak somewhere, we're trying to track it down, but it may break a few times again.



  • Ok, got it running now.

    minor nitpick, in the UI, the top right CPU/Memory might have some rounding issue. I'm seeing values like these: 60.799999999999955 / 290



  • getting this one occasionally:

    [10:25:09 PM][shard0]RangeError: Array buffer allocation failed at new ArrayBuffer (<anonymous>) at typedArrayConstructByLength (<anonymous>) at new Float64Array (native) at new Heap (__runtime__:33218:27) at Object.findRoute (__runtime__:33024:13) at Object.findExit (__runtime__:33114:30) at Object.findExitTo (__runtime__:14667:29) at Object.findPathTo (__runtime__:14831:32) at Object.moveTo (__runtime__:36365:29)



  • @ags131 said in PTR Changelog 2018-01-18: isolated VM:

    @mrfaul the global still behaves like it did before, it can reset at any point. So think of it like ephemeral storage, it can be used, but don't solely rely on it for permanent data.

    I think it would be nice if the whole memory was automatically cached to global in this case. You'd pay for the stringify at the end of the tick every time, but the parse should only be on a reset. If there's one global instance and this "just works" I think it should be in the backend rather than user-implemented.


  • Dev Team

    @tigga This is a nice idea, we'll look into what we can do.


  • YP

    @tigga you can already do that:

    let memory: any;
    let lastTime: number = 0;
    
    export function load_memory() {
    
      // const start = Game.cpu.getUsed();
      if (lastTime && memory && Game.time === (lastTime + 1)) {
        delete global.Memory;
        global.Memory = memory;
        RawMemory._parsed = memory;
        // console.log("Tick has same GID!");
      } else {
        // noinspection TsLint
        Memory.rooms;
        memory = RawMemory._parsed;
      }
      lastTime = Game.time;
      // const end = Game.cpu.getUsed();
      // console.log("memory loaded", lastTime, end - start);
    }
    

    this snippet is from slack, not sure who posted it ,) this also will work on the live server, when 2 ticks run on the same server, which happens quite often.



  • @w4rl0ck I'd seen that before and my post reminded me of it as well! I actually implemented it today (with some help from @PostCrafter). IIUC it's a bit hacky because _parsed isn't part of the spec.


  • Culture

    PTR seems to be running really slowly right now, which is making testing harder.


  • Dev Team

    Native PathFinder is now enabled in isolated-vm server nodes.



  • PathFinder seems to be fine, but there are relatively rare errors, see below:

    Spoiler:
    RangeError: Array buffer allocation failed
        at new ArrayBuffer ()
        at typedArrayConstructByLength ()
        at new Float64Array (native)
        at new Heap (:33344:27)
        // ======= ^ Screeps space
        at Object.findRoute (:33150:13)
        at Object.Creep.moveTo2 (__prototypes:1130:34)
        at Role.work (scout:179:27)
        at legacy_loop (main:1328:60)
        at LEGACY.run (main:1425:9)
        at Kernel:44:77
        // ======= ^ userspace
    

    and

    RangeError: Array buffer allocation failed
        at new ArrayBuffer ()
        at typedArrayConstructByLength ()
        at new Uint8Array (native)
        at new  (:39248:22)
        // ======= ^ Screeps space
        at staticRoomCallback (__prototypes:356:21)
        at global.civilRoomCallback (__prototypes:421:21)
        at roomCallback (__prototypes:1151:24)
        at :33018:31
        at Object.exports.search (:33030:19)
        at Object.search (:39286:38)
        // ======= ^ userspace
    

    Moreover, sometimes V8 heap overflow errors appear unexpectedly, cannot track them for now.