4. raise the 1mb limit on code

raise the 1mb limit on code

  • Culture

    So, since we can't upload our own lodash anymore, are you planning on a versioning mechanism so that we can opt in to newer versions of the library?

  • Culture

    I'm also going to point out that in 2014 there were no minerals, boosts, power, terminals or market. There were no power creeps, which sound like they are going to require different code for each "power part". The game has grown, and so has the code it's required to run.


  • If RAM is a problem, then perhaps it could be counted in the CPU limit? I also support raising the limit to 10 MB.

  • Dev Team

    Can you show us some numbers or something to show that it will affect things this muchI

    I suggest you to wait until the open source server is released, and profile things yourself. And probably help with optimizations if you wish.

    So, since we can’t upload our own lodash anymore?

    Why “anymore”? What has changed?

    are you planning on a versioning mechanism so that we can opt in to newer versions of the library

    Sure, it’s on our Roadmap.

    I’m also going to point out that in 2014 there were no minerals, boosts, power, terminals or market. There were no power creeps, which sound like they are going to require different code for each “power part”. The game has grown, and so has the code it’s required to run.

    I’m not saying the limit will never grow. I mean it’s very hard to simply increase it right away due to current hardware memory limits. I’m sure we’ll find the solution in the future. 10 MB is quite unrealistic though.

  • Culture

    > Why “anymore”? What has changed?

    Lodash is over 500kb minified. My code is rapidly approaching that size myself. Were I to take a dependency on lodash 4 now (or worse yet, if I already _had_) then I would have some very interesting problems in a couple of months (probably shortly after power creeps dropped).

    e: but I missed that that  was on the roadmap, so thanks for that at least.

  • Culture

    2mb would be great to start. Even 1.5mb to get started without causing a major issue on the server. We're happy to discuss/work around the number, but the 1mb limit is becoming an issue.

    > I suggest you to wait until the open source server is released, and profile things yourself. And probably help with optimizations if you wish.

    Yes, we all plan on working on the open source server to optimize things. That's why we're excited. However, you yourself said you're only releasing the game engine, not the infrastructure around it, so it's going to be difficult for us to benchmark your system.

    > Why “anymore”? What has changed?

    What's changed is that she's no longer GCL2 and has to write code for other things. Boosts alone take a solid amount of code.

     

  • Dev Team

    2mb would be great to start. Even 1.5mb to get started without causing a major issue on the server. We’re happy to discuss/work around the number, but the 1mb limit is becoming an issue.

    Well, we need to check how many players are close to this 1 MB limit. 2 MB seems feasible.

    Yes, we all plan on working on the open source server to optimize things. That’s why we’re excited. However, you yourself said you’re only releasing the game engine, not the infrastructure around it, so it’s going to be difficult for us to benchmark your system.

    Creating virtual machine context, code parsing and caching, generating runtime environment - these things will be the same. Only database retrieval and synchronization mechanisms will be different.

  • Culture

    2mb would help a lot, and probably push this problem another year or so down the line.


  • As someone who is new and probably has less than 50KB of hand-written JS and no libraries, Grunt or other automation, I'd be curious to know: Is the 1MB limit across all code in Screeps? Per "branch"? Per source file?

    Documenting this limitation in official documentation might be useful. Maybe add it here? http://support.screeps.com/hc/en-us/articles/205960931-Server-side-architecture-overview

  • SUN

    I guess the issue is either fat dependencies or too much data in code form.
    Pbly Lodash is the issue here. (being non gzipped)

    If you need to use a newer version than the one provided, couldn't a simple upgrade of the lodash version do the trick ?
    Artem would that be possible ?

    Also, if you are really crafty (and desperate for lodash sugar) it is possible to compress lodash in a js string in a static file and then uncompress and eval it at runtime.
    e.g. with something like that
    https://github.com/pieroxy/lz-string/blob/master/libs/lz-string.min.js

    To be honest a big part of what I like in this game are the constraints.
    CPU, Memory, number of constructions site etc...
    Each of them is a challenge and raise the 'fun' part of the game, at least for me.
    1MB of source code of is a ridiculously high limit in that regard :s
    If anything I thing it is too high.
    Unless you store data in your code. (e.g. store static part of the map, or Cost Matrix preset for the rooms around your empire)

    What you do with your MB of code is up to you.
    If you choose to throw half of that to load external code, this is a decision you made, I don't see why this would require the limit to change.

    In the spirit of:
    When you hit the CPU limit, the anwser is: optimize your code and/or change your logic.
    And not, ask the dev to increase the CPU limit.

    My 2 cents.


  • I might be unpopular but I agree with Voronoi - I would vote for not increasing limits.

    Yes, I am new to this game but I do have some years spent developing and designing various projects and tools. In business people often forget or cannot afford to think about optimization too much. Let's use this game to take the fun back to programming. Imho significant part of the game should be around searching for new and ever-better solutions for solving problems in the most efficient way. It is coding competition after all 🙂


  • If discussion is still going on here I'll weigh in as well. You're saying "Let's use this game to take the fun back to programming", but for me the fun in programming is using the tools I like to solve problems. With the code limit of 2MB, it's going to exclude the use of certain tools. I can see its use in a practical sense but not in a fun sense. What's fun is also different for different people.


  • Bringing this back up again. We have a 2MB limit, which is more than 1MB.

    I'm bumping up against it now and am considering my options. One of them is a heavy refactor. Another is that the limit gets raised. A third is to store code in segments and eval or some other similar crazy scheme.

    I think I'd prefer the second option where the code size limit is raised 🙂


  • To be honest, I was against raising the limit in the first place.

    If there should be a change, I'm in favor of it being lowered back to 1mb.

    I'm of the opinion that programming within constraints (cpu, memory etc) is part of the game and I find that quite enjoyable.

    What exactly are you importing Tigga in order to bump over the 2mb? Have you considered minifying/trimming your vendors?


  • @atavus said in raise the 1mb limit on code:

    What exactly are you importing Tigga in order to bump over the 2mb? Have you considered minifying/trimming the your vendors?

    I'm not importing anything. Code takes up space! I'm fully automated and decision logic, especially for combat, is not simple. There's a lot of decisions to be made and a lot of factors to consider in each decision.

    With regards to minifying. Should stack traces be readable? I think so. I agree that minifying is a solution, but then we're not really programming against constraints are we? We're just making debugging harder just to fit within a limit. I agree that programming against constraints is interesting in general, but I don't find code size an interesting constraint at all.

    I'm kinda suprised that it's as low as 2MB, but maybe I don't understand the full end to end cost? 2MB is very small compared to our heap, for example and AFAIK it's only ever touched on a global reset which, with IVM, should be very rare.


  • I'm entirely with you on the stack trace part. I don't minify my code either and have an eslint rule to have all functions named just for clear traces. I suppose I'm just surprised.

    To be honest, I don't have anything comparable to your AI's structure/behavior. My old AI (Andromeda) was built on different principles and had 12k lines (400kb) in production. The new AI (Hydra) is still quite young and only has 7k lines (250kb). It will probably grow to some 20k lines, but I don't expect something above that.

    Your AI is one of the most advanced I've seen. You're probably the first to hit the memory limit "naturally". If you check this thread, you'll see that when people were getting to 1mb it was by importing external dependencies such as newer lodash.

    A few years ago we were doing polls for code sizes and the most advanced code bases were still only around 20k.

    I must admit, I'm glad to see AIs like yours that have reached such level of complexity that they actually touch the memory limit.


  • I think, a bit like Tigga, that the code limit of 2MB is really low, particularly with the feature set (power creeps, event logs, tunnels) getting bigger and bigger. With my workflow atm (sometimes when not at home is use browser access), it would be a pain for coding to minify the code (shortest variable names etc.) I try do automate everything too (code runs bot-like), and - while way behind Tiggas bot abilitys - i am already at 1.2MB and i have nearly no combat code (enemy tracking/some calculation and some simple operation, but nothing like automated room attacks) and many things need to be looked at/optimized as i coded them in a hurry, cause my rooms grew quicker than my understanding of the gamemechanics. Additionally, i use no libs or likewise (all kBs are just code). My base layout - for beeing flexible - is alone 2000 lines (which takes100kB Code-space) without some extern functions for min-cut /wall rampart finding). I am "only" playing screeps since about 9 months, but as i have also a family etc, i have not always that much time for it; still i see my code getting nearer to the limit of 2MB. I do not think that i can get every missing feature (long to do list still) and all new ones into that limit. As far as i see, advanced military will need tons of lines.


  • Right. I'm probably going to be fine on 2MB for a little while longer. As soon as power creeps drop, however, it's just not going to work for me. I'd probably have to give up on stacktraces and run minified, which kinda sucks 😞


  • Some people were poking through the open source driver code and it seems that the user code is sent to the runners every tick: https://github.com/screeps/driver/blob/master/lib/runtime/data.js#L113

    I don't know what the full pipeline for user code is, but if this was changed to only send on code update this seems like it may save a lot of resources and allow for larger codebases.

    ☝

  • yeah, the runners should know whether they will need to perform a global reset on the next tick before pushing their updates to the DB. if it were to add a REQ_PLAYER_CODE flag or something, then the DB would only need to push the code to the runners once per global reset, not every tick, reducing a crap ton of the bandwidth which was one of the initial arguments against raising it 2 years ago. If the DBs are the limiting factor for tick times, then not requesting and sending code for every player every tick should be a major boon to tick times too...