Draft: Store prototype API

artch

In the new Factories update we're going to introduce 40 new resources types. This will increase complexity of both engine code and player code. We think it is time to revisit and refactor both database decisions and game APIs that work with resource entities.

Database

In the database, we will shift from storing all resource properties as root properties in the parent object document, to a unified store object property. It will affect both official MongoDB database schema, and private server schema:

{
  _id: 'XXX',
  type: 'storage',
  store: {
    energy: 1000,
    power: 1000,
    XGH2O: 1000
  }
}

This schema will be used for all game objects capable to store any kind of resources - not only with general-purpose stores (like storages and terminals), but also spawns, extensions, labs, nukers, creeps, power creeps. The property will always be named store regardless of the object type.

Why do we tell you about inner database schema changes? Because if you have a mod that works with private server database directly, you will have to change it accordingly starting from the server version 4.x.x.

Game API

We're going to implement new global prototype called Store and use it in all properties such as StructureStorage.store, StructureSpawn.store, Creep.store (Creep.carry will remain as a deprecated alias), etc. This change is supposed to be non-breaking and backwards compatible: all resources can be addressed as object properties as before creep.store[RESOURCE_ENERGY]. But it will also have some methods for more functionality:

Store.getCapacity([resourceType])
- resourceType is undefined: returns total capacity for general purpose stores or null for limited stores;
- resourceType is defined: returns capacity for that particular resource (for general purpose stores equals to total capacity) or null if this resource cannot be stored.

Store.getUsedCapacity([resourceType])
- resourceType is undefined: equals to _.sum(store) with internal caching or null if this is not a general purpose store;
- resourceType is defined: returns used capacity for that particular resource or null if this resource cannot be stored in this particular structure

Store.getFreeCapacity([resourceType]) - equals to getCapacity - getUsedCapacity.
Store.isWithdrawable() - a boolean flag whether you are able to withdraw from this game object.

This new API is going to remove all inconsistencies and ambiguities that we currently have in different game objects that complicate dealing with resources too much.

The only breaking change will be that Store.energy (unlike StructureSpawn.energy!) will be undefined instead of 0 if there is no energy, this change is long due, since energy is an ugly exception there. UPD: See another option below.

Old-style properties like StructureSpawn.energy and energyCapacity will also remain for backwards compatibility (but deprecated).

We will deploy this change to the PTR with factories simultaneously, and allow for a month or two to test everything on the PTR (and ptr branch on npm), so you can test your game code and mods.

What do you think can be improved further in the proposed change?

TuN9aN0

I don't really like StructureSpawn.store or StructureExtension.store. If it's just energy please consider continuing support as it is now through the energy field. This will also prevent breaking everyone's code in many many places. For labs and nukers I would prefer the same, it's not 'storage', even though you can withdraw if you must from spawns and extension, it's energy to operate with.

artch

@tun9an0 They have stores capable to store only one kind of resources: energy. Making one unified prototype for all game objects will simplify the API to understand and work with a lot, you will be able to reuse a lot of code for different types of game objects.

This will also prevent breaking everyone's code in many many places.

What do you mean?

TuN9aN0

@artch Nevermind I missed the sentence about StructureSpawn.energy and energyCapacity. I think deprecation is not needed though as they are semantically better then a store object with one always there energy entry. It's not that things are suddenly becoming dynamic.

TuN9aN0

How would store.getFreeCapacity work for say a nuker where the energy and ghodium capacity are in their own bucket?

Xenofix

I like the idea of a unified Store-object, but only for Structure, Terminal, Creep and PowerCreep. But for 'other structures' it would be nice to have a unified way to access resources.

Almost every structure depends on energy, so it's good to be never undefined. energy===0 is an ugly exception in store, but it's not ugly in 'other structures' like extensions, towers, spawns, etc.

But every player can change the prototypes to get the desired behaviour, so it's not a big deal.

I'm worried that a preinitialized store for 'other structures' just increases memory consumption a lot more, which is already an issue for some players.

artch

@tun9an0 It's still to be decided. I see two options:

One store bucket for both resources (and you will be able to load only 5000 units of ghodium in it)
Two separate stores energyStore and ghodiumStore each with its own capacity.

artch

@xenofix Yes, implementing Store prototype only for general-purpose stores is also an option which we consider. But it will not allow for massive convenience improvement in the API for transfer/withdraw/whatever code reuse cases.

Almost every structure depends on energy, so it's good to be never undefined. energy===0 is an ugly exception in store, but it's not ugly in 'other structures' like extensions, towers, spawns, etc.

It will be undefined only in the Store prototype, but not in StructureSpawn.energy for example.

artch

OK, another idea came to my mind (top post updated): we can make Store prototype have getters for every resource types which will always return 0 for any resource, and never return undefined at all, i.e. we fix the inconsistency in favor of energy rather than in favor of non-energy:

storage.store[RESOURCE_ENERGY] === 0 // true
storage.store[RESOURCE_HYDROGEN] === 0 // true

Enumeration will work only through properties that are greater than 0.

TuN9aN0

@artch I do appreciate the infrastructure concern but I do think it's kind of leaking here, I wouldn't model this as a store as it's obviously different from say a storage or container. Most of my code dealing with energy is kind of ortoganal to dealing with non-energy as is, checking structure.energy versus structure.store.energy otherwise is just shifting thing about. Another concern is store.energy though, the API currently guarentees .energy to be set. I am sure like me there will be a lot of people who deal with it like this also. undefined < 10 returns false, this will send some quite a few people astray I am sure. I will need to dig through a lot of code to get this fixed everywhere..

TuN9aN0

@artch How many resources were factories going to add again? I sure hope for the Game Object's footprint on the heap it will come through some proxy, otherwise each Structure that can potentially hold something will have this massive hash with 0 values in it...

Xenofix

@artch said in Draft: Store prototype API:

Two separate stores energyStore and ghodiumStore each with its own capacity.

With any other property than store it's not a simple unified API anymore. It has to be one store or no store, but we don't win anything if we wrap number fields in xxxStore-objects.

For developing I must disagree with making the energy and energyCapacity deprecated. If you know the structure you are using, then these fields are the preferred way to access these values. The function of store is just to simplify some code for generic transfer/withdraw.

I like the idea with store-properties returning 0. I think what @TuN9aN0 pointed out is an implementation detail here. I would be against Proxy. Store-objects for Extension or Tower could be using a Prototype which is just a view to the Structure without own memory, which has a property defined in the prototype to be const 0 for each resourceType and energy is a getter to the (hopefully not deprecated) energy field of structure.

artch

@tun9an0 Database and API changes are irrelevant. They introduce separate optimizations for engine code and for player code, but they are not related to each other, we might deploy them in separate updates if we want to, but it seems logical to deploy them together. There is no "leaking" in that regard.

As to your compatibility concern, please see the post above.

How many resources were factories going to add again? I sure hope for the Game Object's footprint on the heap it will come through some proxy, otherwise each Structure that can potentially hold something will have this massive hash with 0 values in it...

40 new resource types. But there is no real data for absent resources in a Store. Store.prototype will just contain a getter for every resource type which returns 0 if the underlying DB data object doesn't have the corresponding resource property.

artch

@xenofix said in Draft: Store prototype API:

For developing I must disagree with making the energy and energyCapacity deprecated. If you know the structure you are using, then these fields are the preferred way to access these values.

I don't get it, why exactly is extension.energy so much more convenient than extension.store.energy that we have to keep this ugly duplication forever?

TuN9aN0

@artch Ok well if it's not for infrastructure concerns, then my feedback is I do not need this change, I think it's currently fine. In my view it actually introduces a leaky abstraction. I don't need unified access to things that are essentially different logically. I would not model it this way.

artch

@tun9an0 Very well then, thanks for your feedback! Let's see what other people will say, since this inconsistency and ambiguity in how resources are stored is being long complained by many players.

Xenofix

To me, carry is inconsistent. I would want to have it be named store instead. And a Store-object for general purpose stores is also very welcomed.

But for every other non-general-purpose structure this is only a little nice-to-have feature to simplify withdraw/transfer. If dealing with structure-specific code, I would prefer to access energy and other fields directly. If it's general purpose refiller/robbery code, then I would go for store.

Important to be is that such a change does not have a negative performance impact.

artch

@xenofix

If dealing with structure-specific code, I would prefer to access energy and other fields directly.

Still can't seem to understand what's the difference between addressing .energy and .store.energy?

As to performance, CPU-wise it will not be affected, only heap-wise concern is valid here, and we can simply raise the heap limit on deploying this change.

ags131

As both a mod developer and player I'm exited about this update, for modding, it was always annoying not being able to iterate over resources in a simple way. As a player, the entire store/carry/energyCapacity/carryCapacity mess was annoying, in some cases, I actually wrapped all those to make them match, very similar to how this is designed. It might seem a bit more clunky for stuff with only one resource, but simplifies more than enough other code to make it worth it.

For those wanting to keep energy/energyCapacity, they can easily do a simple prototype extension in their code. Object.defineProperty(StructureExtension.prototype, 'energy', { get() { return this.store.energy } }) Object.defineProperty(StructureExtension.prototype, 'energyCapacity', { get() { return this.store.capacity } })

wtfrank

@ags131 yeah I've had to abstract around energy, store etc in various places (e.g. if (instanceof_energy_structure(struc)) { } else if (instanceof_store_structure(struc)) { } )

Before I used typescript and enforced this at "compile-time", I remember fixing various bugs where I tried to treat an extension as if it had a store, or some other structure the other way around.

I will be glad to have a unified interface, even though I'll have to at some point rework some of my code.

It will make the game slightly less complex for new players.