4. Native Folder Support

Native Folder Support


  • There's been some discussion in the past of a desire for support for native folders in our codebases.

    It seems this conversation always devolves into "just use grunt/webpack/gulp/rollup". But I don't want to do that, and we can skip that line of thinking entirely this time. What I'm hoping to achieve with this thread is to discuss the feasibility of adding this feature. If it's not possible, I'd like to know why. In the past I've gotten answers like "it's complicated". Well, I'm hoping we can elaborate on that.

    What kind of complications do we face?

    • require needs to support it. (We already patch that, seems like an easy enough change)
    • How would it affect wasm modules? (This is sandboxed as well, right?)
    • Would need to work with github integration (Probably a minor change to how it's cloned)
    • Would need to work with steam client upload (Filesystem is already being watched)
    • [This list may grow. Give me suggestions, or correct me where I'm wrong]

    Screeps is often open to PRs, but we don't have access to the github integration or client frontend code for obvious security reasons, so this would have to be an official project. But I'm not convinced yet this would be difficult to add. I'd like to discuss it, as this would make our codebases significantly more manageable. What would it take?


  • The minimum viable solution isn't a huge change for the engine. Just allow files to be uploaded with '/' in the name. Then have frontend render the files as a tree.

    It gets harder is if you want to be able to require('./other-role') from within a roles folder, worse still require('../utils/foo'). Require needs to be patched to create a virtual filesystem to support this completely. Alternatively some clever regexes can cover 99% of the cases and rest could be not supported.

    1 Reply

  • @deft-code Thank you for your input.

    I hadn't considered the filesystem rendering in the gui. That might be the most complicated part.

    I've since learned the code is stored in the database, scoped to a user. So permissions aren't really a problem here. You can't accidentally stumble into somebody else's file space. As for the relative pathing, nodejs has a path module with some functions that would help here. path.resolve for instance could calculate an absolute path name from the string which could then be run against the require.cache or sent to lookup the key from the database. Should actually be fairly painless.


  • It looks like in order to load them we'd need to touch driver.evalCode and engine's requireFn to make a new requireFn for each module (far as I'm aware it's the only way to keep track of the current filepath for the module). This would happen only when a module is a loaded. Cache hits would return the existing module still. Path resolution would occur each call to require(), so there would be a little extra overhead but I don't think it'd be significant.

    To further complicate this, we can't use node native modules directly in the engine, so we might have to write our own copy of the path functions we want to use. ..Annoying but hardly the end of the world.

    I'm toying with some code but ultimately it's an experiment or proof-of-concept, and would require the devs to implement the gui and code-upload portions.


  • It's possible to use the call stack to determine which file you're in. https://github.com/deft-code/typescreeps/blob/master/src/debug.ts#L9 This is very similar to the code I've using now. I use this trick to add a file and line number to my console.log messages.

    Using this trick you could write an alternative require proof of concept entirely in game code rather than in the engine. You might have use a different separator; I don't know what the api will do with files with / in the name. (the current require implementation ignores leading ./ so might have to work around that also.