4. Possible solution for multi language support?

Possible solution for multi language support?


  • Some (most?) of the public bot codebases are using TypeScript already, and there's a lot of folks using WASM transpiled stuff.

    JavaScript as a target is becoming more and more common due to it's use on browsers, so I just don't think this is an issue really versus the entrenched playerbase that a titanic shift in language support would throw out the window whole-cloth.


  • That change would be made in the background and not disturb the current scripts.
    Hence the conversation of the current API to a interface. Which means everything should work as expected besides the scripts with highly modified environments which really get to the guts.

    And no this doesn't add a barrier. CodingGame.com uses this system and it works great.
    Stin/out is basic functionality which every coder/scripter should know how to use.
    WASM on the other hand has some really nasty particularities which I would describe as barrier and you can't really call it finalized either.

    I admit that any interpreted language with slow interpreters would have a disadvantage if they are not cached. But there are tons of upsides for stand alone executables like proper thread control of a script.

    This game needs a bigger audience but there are a lot of potential "professional" players that are reluctant to try it because they don't like or can't be bothered with JavaScript...


  • @artch could please move it back?
    Under no angle is this a feature request.

  • Dev Team

    You're offering a major refactor in order to introduce some new capabilities, hence this is a feature request, I believe.

    As for the idea itself, it's very, very technically complicated matter. You probably don't see the entire picture of implementing separate properly isolated virtual machines in different languages working in one server-side cluster simultaneously and synchronously. CodingGame can't be compared, since their architecture is totally different, they don't have the persistent world aspect. As much as we want it, not going to happen at least in 2018, sorry.


  • I'm not talking about VMs in different languages, I'm talking about isolated threads.(which share a lot of similarly, but aren't a VM)
    And yes right now there are more pressing matters for you. I see this as discussion to achieve a optimal and sustainable solution for this and not like a feature request but that is a matter of interpretation 😉

    About the whole picture, yes I see it and the biggest bottle neck for screeps right now in my understanding is the database connection and its load.

    With the decupling from the scripts of the engine (which is essentially what the isolated VMs currently in beta are doing) you gain a huge advantage.
    This allows proper metrics with that load balancing becomes a breeze.
    Changing to the stin/out API would even further that making both engine and scripts complete separate entities which could in theory run in different universes as long they are able to communicate.
    Would offer a immense flexibility.

  • Dev Team

    How do you suggest running userland code in different languages on servers?

    isolated-vm has nothing to do with this matter by the way, it's just a replacement for Nodejs vm module.


  • Oh, crap didn't see your reply.
    And yes I'm talking about userland code but thinking about it why don't you offer automated WASM compilation for some of the most popular languages. This could help a lot.
    I really like screeps I just don't want to see it die of a lack of non-paying users.
    (Yes, I'm currently one them but my current "duck-taped script" isn't worth the money -_-)

  • Culture

    I've actually had a system like this working at one point, except it was using TCP rather than stdio, it would be a massive project to properly sandbox it, likely requiring containers for each user, which would be difficult to manage automatically. As @artch mentioned, CodingGaming is a completely different setup, they do a onetime container for each run, and unlike screeps, its not time sensitive so doesn't have to respond as quickly.

  • YP

    I thought about it some time ago and my idea was using redis as the interface to other runtimes. Maybe that's possible, but as @ags131 said permormance is key.. and the screeps dev are currently investing lots of time in optimizing the runtime ... I don't think new runtimes that might cause slower tick times are currently the right direction.

    1 Reply
  • Dev Team

    @w4rl0ck Exactly. Implementing a protocol between runtimes in different languages and processors is trivial, it can be done in a few days easily. Discussing its design here (whether to do it via stdin/stdout, Redis, TCP, etc) isn't worthwhile, it's not a complicated engineering task in any sense, it's a basic one. It is implementing proper runtimes for each language that is complicated, it would require months of R&D for each language, and we're really not ready to start this journey right now.


  • I don´t know if I was understood correctly I´m talking about to skip the "runtime environment" completely and to run standalone executables.
    They don´t even need necessarily a VM to do so, just super strict right management and bare bone libs.

    If you compile those "on side" it should be fairly easy to scan for malicious intends.
    There should be a number of such sandbox compiler already out there.

    But this doesn't help with interpreted languages like Lua those would still be in need of a proper runtime environment.

    1 Reply
  • Dev Team

    @mrfaul In order for your idea to work we'd need to run a separate container for every user to manage their resources usage, with instant startup, in an environment of thousands synchronized users. So my point still stands - the complexity of such a system makes it not even a major refactor, but a huge R&D project with uncertain results.