Optimizations roadmap
-
Last time I used mongoDB (ditched for the same reason) it was really good as storing text gibberish, and rubbish for anything else. Maybe mongo is good because of the serialized memory and the code storage. But it was really quite slow on normal stuff back when I used it last.
-
Have you guys considered sharding based off of data types instead of rooms or worlds?
Another thing is sharding off of primary key.Not only considered, but even tried and benchmarked in production.
Will the isolated vm module made available for private servers ?
I hope so.
I have seen benchmarks of postgres jsonb storage beeing faster in storing and manipulating json objects then mongo and some data would also fit to be stored in relational tables.
We did benchmarks of Postgres JSON and JSONB storage on our workload profile - it doesn’t provide any significant performance increase in multi-threaded environment compared to WiredTiger in MongoDB 3.2+ with document-level locks.
-
> Not only considered, but even tried and benchmarked in production.
What happened, why was this abandoned?
I just feel that there are ways to shard the database without requiring complete different worlds with their own separate tick times. Divergent tick times make things a lot more complicated, and not in a good way.
-
I am concerned with sharding for a few balance reasons, e.g.;
- Depending on accessibility, players on a faster shard get an advantage over players on a slower shard. If a portal is open, the player on the fast shard can mine more power, harvest more energy and spawn more creeps than a player on the other side, giving a serious advantage.
- It adds another player of complexity for smaller players to overcome when setting up multiple rooms / spread rooms.
- Presumably respawn zones will often be in shards, to drive players off the main shard to decrease tick times - how much will this stagnate the main map?
Otherwise seems cool.
-
I'm not so keen on the shards system, how will subscription work between them you said they could have differing CPU limits. It could split up the game.
-
One of the aspects that makes Screeps such an interesting game in the single world. Much like Eve online it creates a political environment that is exciting. I'm concerned sharding the game will fracture this, and lessen the overall political scene in the game.
I'm interested to see how this idea develops, but really I hope you figure out a way to shard the game without separate worlds.
-
Since it's gotten lost in the discussion about the shards I did want to jump in and say that that the other items look absolutely fantastic.
I think the new VM solution is great, and I can't want to no longer have to deal with bad nodes and other performance issues (with the added bonuses of our state moving around from node to node).
WebGL is also super exciting. I've seen some great demos that looked fantastic and performed beautifully. This upgrade is going to be great.
-
The new VM ideas are amazing! One single global will surely lower the tick times as well.
For example:
- It could save the CPU used on the Memory serialization, which is about 4% of my overal script times (higher in global reset ticks)
- It can easily terminate misbehaving players, or
- Omit them if they don’t finish in a certain amount of time
- Keep static game objects the same every tick evading
- Avoid having to do
register.wrapFnfor every method declaration
I think there are a lot of other things that would make life easier if we get t keep the exact same global across ticks.
World shards are quite an interesting idea. I imagine them being a building which can transport creeps to another planet. The success of it depends gravely on it’s implementation. We could be able to build “portal hubs” but only 1 per player (globally owned structure). I don’t think “random portals to another shard” is a good idea.
It could also heighten the CPU-cap per player again!
We would only be able to user 300 CPU per shard, if your GCL is above 27 you have no choice but to expand to a different zone.The CPU limit for each shard can be set on a special page
If we’re going this way, allowing script access to these things is vital. I’d love to see a more thought-out plan quite soon. No need to speculate on how the memory will be saved in different shards.
-
What are your plans for the existing world should 'shards' be implemented?
I understand it will depend on the implementation, but is the world going to divided up or reset? (Or neither? (Side by side with a mass migration? that'd be cool))
-
What happened, why was this abandoned?
I just feel that there are ways to shard the database without requiring complete different worlds with their own separate tick times. Divergent tick times make things a lot more complicated, and not in a good way.
Believe me, we tried all kinds of good and black magic regarding classic database sharding. It just doesn’t provide the desired level of scalability in comparison to this architecture shift. We are quite positive that world sharding is more correct way to implement horizontal scalability than database sharding.
Depending on accessibility, players on a faster shard get an advantage over players on a slower shard. If a portal is open, the player on the fast shard can mine more power, harvest more energy and spawn more creeps than a player on the other side, giving a serious advantage.
This is a temporary issue. Eventually shards will become self-balanced, when more and more players would want to benefit from the faster shard, making it slower and the old shard faster.
It adds another player of complexity for smaller players to overcome when setting up multiple rooms / spread rooms.
We predict that only a few percent of players will be up to establishing cross-shard colonies.
Presumably respawn zones will often be in shards, to drive players off the main shard to decrease tick times - how much will this stagnate the main map?
Both respawn and novice areas will still be created at every shard.
One of the aspects that makes Screeps such an interesting game in the single world. Much like Eve online it creates a political environment that is exciting. I’m concerned sharding the game will fracture this, and lessen the overall political scene in the game.
I’m interested to see how this idea develops, but really I hope you figure out a way to shard the game without separate worlds.
The world is still single, just consists of connected shards. Eve Online is actually a great example of this, since they implemented the same idea of different game speed in different regions of space - so called “time dilation” mechanic.
We would only be able to user 300 CPU per shard, if your GCL is above 27 you have no choice but to expand to a different zone.
CPU limit is per account, not per shard. Say, if you have GCL 28, and 100 CPU allocated to Shard1, then you have only 200 CPU in Shard2.
What are your plans for the existing world should ‘shards’ be implemented?
The existing world will become the first shard. Nothing else will happen to it.
-
> I imagine them being a building which can transport creeps to another planet. The success of it depends gravely on it’s implementation. We could be able to build “portal hubs” but only 1 per player (globally owned structure). I don’t think “random portals to another shard” is a good idea.
I actually think that the new shard should be easier to get to than that, and that the entry/exit points should be in neutral rooms (perhaps in the same rooms as the NPC terminals). If the other world is going to have higher tick rates then people on the old world are already being punished- making it harder to access the faster world is just salt in the wound.
I really think the shard idea would be easier to deal with if the tick rates and time were kept close together, even if it does mean throttling the new shard when it gets too far ahead.
-
> We predict that only a few percent of players will be up to establishing cross-shard colonies.
> The world is still single, just consists of connected shards.
What it sounds like is that the world will in theory be a unified world but in practice will actually be separate worlds.
> This is a temporary issue. Eventually shards will become self-balanced, when more and more players would want to benefit from the faster shard, making it slower and the old shard faster.
The old world has a lot of baggage- zombie players on 10cpu, or just somewhat inactive players who don't mind paying but also don't do anything. I don't think relying on it eventually balancing it out is the best way to handle it. New players will naturally spawn to the new server but older players (the ones who have supported the game the most) have established themselves and many of them will not make likely make the jump. I don't think the shards are likely to balance out soon.
I think the sharding idea can be a good mechanic, but I really think that the "single world" concept needs to be embraced. To me that means tick speeds mostly synchronized (there can be variation, but it should average out to about the same) and easy travel between worlds. In my mental model of the screeps world I'm viewing shards as another dimension- a Z axis to go with the X and Y (or in this case north and south) and would like to see "elevators" in all the highway intersection rooms for going between levels.
-
So, how shall the market work between shards?
-
The old world has a lot of baggage- zombie players on 10cpu, or just somewhat inactive players who don’t mind paying but also don’t do anything.
Scripts of players without subscription stop working after 30 days of inactivity (no log into the game during 30 days).
I don’t think relying on it eventually balancing it out is the best way to handle it.
If it doesn’t balance itself, then we’ll start to balance it manually. Eventually our goal is to provide lower tick rates for all shards.
In my mental model of the screeps world I’m viewing shards as another dimension- a Z axis to go with the X and Y (or in this case north and south) and would like to see “elevators” in all the highway intersection rooms for going between levels.
Exactly, you got the idea of it. I can’t tell for now where those portals will be located, but what I can tell is that they will be neutral, and there will be plenty of them so that everyone can access. Inter-shard portals can be considered just another type of inter-room exit tiles. As long as all interested players can easily move back and forth, it remains the same idea of a single world consisting of isolated game zones (rooms/shards).
-
> So, how shall the market work between shards?
They would be seperated... otherwise the economy of a new shard would already broken from the beginning
> I really think the shard idea would be easier to deal with if the tick rates and time were kept close together, even if it does mean throttling the new shard when it gets too far ahead.
But the lower tick rate on the new shard is one of the only things I can think of why you would want to spawn on a new shard.
> The world is still single, just consists of connected shards. Eve Online is actually a great example of this, since they implemented the same idea of different game speed in different regions of space - so called “time dilation” mechanic.
It's a bit like apples and pears ... the time in eve online is only slowered in the star systems where currently a big combat is happening ... it has no impact on the econoly... and it is one of the most hated systems in eve online
It more reminds me on a mmog I played several years ago .. I currently don't remember the name. players and guilds could create villages and build cities and one of the features was that you can move your guild to other servers. Later there were guilds that were not interested in building cities but to only move from server to server and destroy others stuff
-
So, how shall the market work between shards?
They will be fully isolated. But you can start doing some import/export shenanigans between them, might be quite interesting way to raise some credits
-
I think the portals will have to work a bit differently. The room bouncing problem is already bad enough when your script is running on one server, but having to send a creep and have another script grab it at the right tick sounds like an amazingly frustrating experience. Having "entrance" and "exit" portals for each side, so a creep can get on a square in one world and end up on a non-portal square in another, with no concern about bouncing between the two.
I also think Memory should be completely independent so one server doesn't overwrite the other. Being able to request segments from different shards would give a useful communication channel.
-
> But the lower tick rate on the new shard is one of the only things I can think of why you would want to spawn on a new shard.
If the worlds are connected in a stable way then I would happily put an outpost on the other server, and build up on both sides. Having bases that players can't get to without either respawning or investing in extra-dimensional portal traversal seems like a huge strategic advantage.
-
> They will be fully isolated. But you can start doing some import/export shenanigans between them, might be quite interesting way to raise some credits
Credits too? Will subscription tokens be the same across shards?
-
I think the portals will have to work a bit differently. The room bouncing problem is already bad enough when your script is running on one server, but having to send a creep and have another script grab it at the right tick sounds like an amazingly frustrating experience. Having “entrance” and “exit” portals for each side, so a creep can get on a square in one world and end up on a non-portal square in another, with no concern about bouncing between the two.
Activating a portal would most likely require some explicit action like
Creep.moveToShard(portal, shardName)since the same portals may be used for more than two shards in the future.I also think Memory should be completely independent so one server doesn’t overwrite the other. Being able to request segments from different shards would give a useful communication channel.
Memory and segments will be independent to prevent race conditions, and due to loosely coupled architecture (runtime workers will be connected only to their shard’s database, not to others’). We still need to figure out some way to pass messages between shards in an asynchronous manner though.
Credits too? Will subscription tokens be the same across shards?
Can’t tell anything for sure now. We need to prevent race conditions on operations with credits and other account-wise resources like tokens. It can be achieved by either introducing locks and mutexes on such operations (reducing the performance), or separating their balances on each shard with an ability to transfer between them.
