Artem - you mean the number of flags?  I'm all in favor of a 1,000 flags per GCL limit, and as far as I know for the people in this thread I have the most flags.  There is a reasonable amount of flags, and name lengths, etc - I think we all agree.  What we are all struggling with is how this was brought to our attention and the proposal seemed to do more harm than good.

So, in my mind, this is a solid proposal for now:

NO added parsing cost initially (This gives people more time to prepare and will allow you to collect metrics on how the other changes affect the servers)

Flag names are limited to 60 (since tedivm was asking for 60) characters

Flag count is limited to 1000 * gcl (this can be revisited later)

How does that sound to you?  I completely agree that there needs to be some limits in place on flags, they are too cheap as they currently are - but the costs incurred by turning this on will be enormous to players and their code.  If we can try the second two points first, then reevaluate where we are, I think we'd all feel better about discussing options for flag costs while not under the pressure of a deadline.

Artem - if I cache object instances in global scope, they persist across my ticks within the same worker node.  Would you be able to assign them to our own personal global context as a data container, then when we access flags the parsing cost is offset by anything that's already in our global cache on the worker?  I'm not sure if that would be easier than trying to handle it outside of the secure execution context.  Does that make things easier?

Artem, if the flag instantiation is the cost, can you please review and comment on my suggestion posted earlier for re-using the instances across ticks?  Thank you.

For the record, i have about 7000 flags. I use them primarily to mark locations of buildings, with a focus on planning and enforcement of buildings. What that means to me is that a certain flag marks a location where a structure should exist, pending a RCL precondition, and then should permanently exist thereafter. This allows me to place structures "in advance" and then build or rebuild them later by marking the location and using the color code of the flag as information for my management. I also use flags as spawning demand "tickets", where a flag is used to request a creep to come "do something" at a location.

The flags documentation states

Flags can be used to mark particular spots in a room.

Having thought this over more, I'm not sure what it is I am doing wrong with flags that I would need to incur a 30++ cpu penalty in the next two weeks. I am marking locations in my empire for things to be built, locations of interest that allow me to navigate between rooms, and I am marking locations for creeps to go do work. If the Flag object is too expensive on the server, provide me with a drop-in alternative, or allow me to build the flag objects myself. I'll implement the cache I suggested for myself, if you let me. That will both reduce load on the server, and I can keep my flags. You've had multiple suggestions from a few people, including myself, on how to reduce server. Can you please address those suggestions even as stop-gap solutions until there are alternative or better options in place?

Artem, what about having flags not be fully instantiated every tick on the server?  You could instantiate them once, then cache the resulting objects at the end of each tick and apply the data changes to the existing objects and add updated flags to Game.flags for the player, and discard the cached list.  This would reduce instantiation cost as the objects would already exist, and you'd instead be iterating over the flags to synchronize the data with the properties.  Is this feasible at all?

In short, promote on write - here's the basic idea:

// Tick ends
_gameCache.flags[playername] = Game.flags
// Tick start
_.forEach(flagroomData, f => { 
  // Split, etc
  if(_gameCache.flags[playername][parsedName]) {
    // Update properties directly 
    _gameCache.flags[playername][parsedname].color = COLOR_RAINBOW;
    Game.flags[parsedname] = _gameCache.flags[playername][parsedname];
  } else {
    Game.flags[parsedname] = new Flag(..parsedinfo);
  }
}

// Clear cache, references to flags that already exist are now in Game.flags for the player
// and will be copied back out on tick end
_gameCache.flags[playername] = {}

That's the problem, though, Artem.  I was certain it was GC and thank you for confirming that, but the GC cost is something that is not necessarily related to our code, but could also come from the game engine, previous players code, or elsewhere - yet we have to pay for it on our own ticks and our own cpu time.  I know that's a hard problem to solve, and my complaint isn't with having to pay it, it's with the inconsistent timing of *everything* when GC hits.  Adding another parsing step to our cpu costs will cause ticks to vary even more wildly than current, and if you have a "bad" tick you can hit the tick limit before you finish processing your code.  Isn't there a way to record aggregate flag parsing cost for each tick, then average it?  You can take that number, add 20%, and call that the new "constant" for per-flag parse time.  You could even make it a moving average - I just think it's unfair to have a 4x difference in CPU cost between ticks and expect us to be able to code around that.  I'm not looking for cheap / free flags again - hell, make the parsing cost a full CONST 0.2 for all I care, but please work with us to find a way to make it consistent.

Using your same benchmarking style of test for Memory on the production systems, I see these results:

[9:05:55 AM] Tick 14065006 Memory parse time result: 30.2501
[9:05:58 AM] Tick 14065007 Memory parse time result: 9.3850
[9:06:00 AM] Tick 14065008 Memory parse time result: 6.9617
[9:06:03 AM] Tick 14065009 Memory parse time result: 9.7145
[9:06:06 AM] Tick 14065010 Memory parse time result: 10.6524
[9:06:09 AM] Tick 14065011 Memory parse time result: 11.7271
[9:06:12 AM] Tick 14065012 Memory parse time result: 7.3918
[9:06:15 AM] Tick 14065013 Memory parse time result: 11.2062
[9:06:18 AM] Tick 14065014 Memory parse time result: 11.3516
[9:06:21 AM] Tick 14065015 Memory parse time result: 26.5043
[9:06:24 AM] Tick 14065016 Memory parse time result: 50.3858
[9:06:27 AM] Tick 14065017 Memory parse time result: 7.6152
[9:06:30 AM] Tick 14065018 Memory parse time result: 11.9079
[9:06:33 AM] Tick 14065019 Memory parse time result: 10.3699
[9:06:36 AM] Tick 14065020 Memory parse time result: 31.6772

Do the production servers also have background processes running?  Because this is a fluctuation between 7.6 cpu and 50.4 cpu, just to access memory.  This is the test code:

module.exports.loop = function () {
    // console.log(`------------------- tick start: ${Game.time} -------------------`);
    let preMemCpu = Game.cpu.getUsed();
    Memory;
    let postCpu = Game.cpu.getUsed() - preMemCpu;
    console.log(`Tick ${Game.time} Memory parse time result: ${postCpu.toFixed(4)}`);
...
}

Let's just call this what it is - variability due to system overhead, maybe garbage collection, I don't know for sure.  If you can see this kind of variability in production, why is PTR written off as an invalid case when seeing those numbers?  I'm happy to post more from production, maybe I'll get lucky again and have the Memory test report 200 cpu as I have seen in the past.  Please take this seriously, because right now it just feels like you are writing our concerns off.  Flag processing plus memory parsing on a single tick could (given only the numbers I've posted in this thread) cost 80 + 50 = 130 cpu, which is my current limit.  

Oh, and some more results from the memory timing test:

[9:14:38 AM] Tick 14065185 Memory parse time result: 9.6542
[9:14:41 AM] Tick 14065186 Memory parse time result: 72.8470
[9:14:44 AM] Tick 14065187 Memory parse time result: 86.3037
[9:14:47 AM] Tick 14065188 Memory parse time result: 17.5044

What is the reason for this?

Is it at all possible to enforce a constant cost on the flag parsing operations?  I know for Memory this isn't possible as the number of objects within and the layout affects things much more than the size, but flags are a rather constant object.  On the PTR the parsing cost is jumping anywhere from 19 to 83 cpu, which is a big difference.  On good ticks the cost is fine, maybe even too low, but on bad ticks the cost is unreasonably high.  What are your thoughts about making flag parsing cost a (low cost) constant amount per flag?  It would make the calculation of impact *much* simpler and they'd still have a per-tick cost without the (sometimes absurd) fluctuations in parsing cost that I already see with Memory.

Also, I don't have any of my flags on the PTR.  I can't test this unless they are copied over, can you please ensure everyone has their flags copied over so we can check the cpu usage and adjust accordingly?

So, as some people may know, I have a lot of flags.  I don't try to use them as a way to get around memory usage but as location markers, which leads me to a very important question - can we override how flag parsing is done similar to how we have RawMemory?  I'm fine with this change going in, flags are definitely a bit too cheap as things stand, but what I really would like to have is the ability to change what happens during the parsing step.  As an example, I don't usually find Game.flags itself useful for more than checking if a flag exists by name, what matters most to me is colors and color combinations.  Currently, I have to process all of my flags in a secondary step to filter them by color - I'd like the ability to do this during the initial parse period as well to save on iteration cost.  If we can add additional functionality to the base Game.flags parsing, I support this change - if we can't, this is a big problem since I'd essentially pay for parsing twice each tick.