PTR Changelog 2017-09-25: WebAssembly support


  • Culture

    I've been testing with wasm and so it seems to perform very well, I'm not sure of the memory usage, but overall CPU usage tends to be less than similar JS code, even with the calls back into JS space. My current testing code is on GitHub, it uses a different approach than the guide above and handles much more of the marshaling of types and helps deal with pointers and such. With this, I'm easily able to spawn and random move over 300 creeps with almost no overhead. If this ends up being available, I wouldn't want to switch all my code over to it, I'm not a C++ guy, but I would implement the more computationally intensive tasks in C++ to reduce CPU. Stuff like distance transform, skeletonization, layout planning, path compression, this can all be done in C++ and reduce overall CPU required.


  • YP

    I'll give it a try now.

    I wonder if webassambly could not only be used in the user side of code ... but maybe also make some of the server code faster ?



  • Nice, this is definitely something I would use if it ever makes it in. Finally my work experience apply to something that matters! :)

    WebAssembly says it does or will eventually support pthreads, OpenGL and other system libraries. Will those be available in screeps and how would pthreads for example impact CPU usage? (haven't had a chance to actually try it on PTR yet)

    Would .wasm file size would count against max script size I assume?


  • Dev Team

    @unfleshedone

    WebAssembly says it does or will eventually support pthreads, OpenGL and other system libraries. Will those be available in screeps and how would pthreads for example impact CPU usage? (haven't had a chance to actually try it on PTR yet)

    All WebAssemply features should backed by Node.js to be supported in Screeps. OpenGL, obviously, doesn't make any sense. pthreads would require Web Workers AFAIK, and it is not supported in Node.js since they are designed for browsers, and Node.js is single-threaded.

    Would .wasm file size would count against max script size I assume?

    Yes, they are stored in the same storage in base64 format (approximately 4/3 of the original binary size).



  • @artch OpenGL might make sense actually -- rendering room map to a texture, running shaders on it (I'm sure many image processing and computer vision algorithms can be done this way) then exposing resulting texture to the main script. Shared tenancy on GPU might be a problem though.

    (Not that I'm saying we need openGL support, but that people will try to use it if it is available, and that's another resource that will need to be charged to CPU)


  • Dev Team

    @unfleshedone OpenGL doesn't make sense merely due to the absence of GPU on our runtime servers.



  • @artch: halp, require('binary') seems to be broken since last PTR patch or smth other update?

    const binary = require('bytecode_file') causes following error:

    wasm:1
    (function __module(module,exports){ [object Object]
                                                ^^^^^^
    SyntaxError: Unexpected identifier
        at main:12:17
        at sigintHandlersWrap (vm.js:92:15)
        at ContextifyScript.Script.runInContext (vm.js:50:12)
    

    I just finished lzw native implementation+porting and gonna try it inside screeps, but got an error.

    BTW: please, don't disable WASM on PTR at least for now. For example, I'm working on proof-of-concepts and gonna release some API and Emscripten bindings examples and tested production tools (such as native lzw en/decoder) soon this week, so I think it will be easier for community to start working with WASM after giving of some complete examples and utilities.


  • Dev Team

    @mototroller Should be fixed now.



  • @artch: yep, awesome, thanks!


  • YP

    rust has now a direct wasm build target : https://www.hellorust.com/news/native-wasm-target.html


  • Dev Team

    Added binary upload support to the in-game IDE:

    This change will be deployed to live servers on December 14 along with upgrade to Node.js 8.



  • @artch

    Yay. Now I need to know two languages to play this game well.

    Really, what is the goal here?

    All this does is make it incredibly painful to figure out how to write a performant AI using a bunch of undocumented features. Great.

    Can we get back to making the actual game better?


  • Dev Team

    @shedletsky You don't need to know other languages, you are able to use them, that's a big difference. JavaScript is performant enough if used properly. But now you have a choice.

    Supporting WebAssembly is just a side effect of upgrading to Node.js 8, which is inevitable since 8.x branch is now LTS (Long Term Support). WebAssembly is a new standard that is widely adopted in JavaScript world, it's not strictly related to C++, you can compile Rust or something else to it, and more languages will be supported soon. This way Screeps can (at last!) enable support of different languages besides JavaScript, and it's actually a good thing, it extends possible target audience. But you surely are not obliged to learn all these (potentially dozens of them) supported languages and are free to choose any particular language you like.

    I would say that supporting WebAssembly and other languages through it may potentially become the biggest thing "making the actual game better".


  • Culture

    The webassembly changes are awesome.

    I have a feeling not a lot of people are going to be programming their code directly in other languages, but I can see people building libraries for improved speed. @Mototroller has already started an open source webasm project, and I can imagine more are coming.

    @artch, I know this probably isn't setup yet but will there be a way to incorporate webassembly in open source bots? I can compile before pushing to NPM so that isn't an issue, but it would be great if we could include these libraries.


  • Dev Team

    @tedivm What do you think would prevent you from doing that? You can include both .wasm files and their sources with instructions how to compile them.


  • Culture

    @artch but once they're compiled what do we do with them? Can we just leave the compiled binaries alongside the regular code and expect it to work?

    I think my confusion was that previous (I thought) they had to be uploaded to a separate branch, but it looks like that's no longer the case.


  • Dev Team

    @tedivm I lost you here, why they have to be uploaded to a separate branch? They are plain binary files which are uploaded to binary Screeps modules via either in-game IDE, grunt-screeps or Steam client. Like in the example above, you have:

    dist/
      main.js
      addTwo.wasm
    

    This corresponds to Screeps modules: 0_1506332804337_chrome_2017-09-25_12-46-19.png


  • Culture

    @artch thanks for clarifying. I had misread this-

    0_1512678788663_Screen Shot 2017-12-07 at 12.32.27 PM.png

    and thought we were uploading binaries to a separate branch on screeps. What you're saying makes for more sense.



  • @mototroller said in PTR Changelog 2017-09-25: WebAssembly support:

    Being inspired by Screeps, I was looking for a scalable, high performance and virtualization-friendly solution for Screeps-like AI environment. The candidates were Lua (provides high performance virtual machine and a simple script language) and WA (extremely high performance + built-in virtualization by design).

    I've been in a similar situation recently and when evaluating WA vs Lua I decided for Lua.

    WA is definitely a big thing, but in it's current state you are in for a lot of trouble. Even if you ignore security risks - and since this thing is basically still alpha you actually really shouldn't - the issues when trying to get a project done with WA are currently too excessive. Eventually WA will become a reasonable alternative, but that's something far in the future.

    The main issue with WA is error handling. Even the best programmer will eventually make a mistake, and at this point you need good debugging and logging to find and fix it. WA makes debugging a nightmare. All tools you are used to from JS debugging will not function, and also all tools you might be used to from C/C++ developing will not function either. You can handle some issues with logging, recompiling and waiting (which is very tedious), but some of the lower-level issues like incorrect memory access, pointer failures, off-by-one errors, all that are a major pain to find and fix when you have zero tools to help you. In the end all you can do is to write extensive test-cases and hope to catch all errors, then spend days debugging those you didn't catch.

    And on top of that most JS programmers are not experienced with these kind of bugs, as they simply do not exist (in that way) in JS. Which means that many people will have to learn a completely new language (and all the issues that come with it) first, and in the worst way possible.

    Lua was definitely the better choice for my kind of problem. While it is not as fast as native C, it's still much faster than JS and therefore suitable for performance-critical parts. On the other hand debugging and writing code is much simpler, as Lua is much closer to JS than C.


Log in to reply
 

Looks like your connection to Screeps Forum was lost, please wait while we try to reconnect.