Allow access to Node's TextDecoder

Reply to Allow access to Node's TextDecoder on Invalid Date

joerichey — Invalid Date

I'm working on getting Screeps to work with Rust's wasm-bindgen (the official way to run Rust in Node.js). The only thing blocking this is that the Rust/JS shim makes use of TextDecoder for performance. This API has existing in Node since Node 8 (screeps uses Node 10) and is also supported in most major browsers, so it would also work with the simulator.

In fact, TextDecoder is already available in the simulator at global scope, so Rust already works there. We just need TextDecoder to also be there when running in Node.

Previous discussions mentioned allowing access to Node's Buffer class, but that's no longer necessary as the Rust Shims now just use TextDecoder and Uint8Array.

This came up because the toolchain that the existing Rust API uses (stdweb) is no longer maintained, and the wasm-bindgen alternative is officially supported by the Rust org.

Reply to Allow access to Node's TextDecoder on Invalid Date

joerichey — Invalid Date

Github Issues:

Relevant APIs:

Reply to Allow access to Node's TextDecoder on Tue, 15 Sep 2020 09:17:07 GMT

joerichey — Tue, 15 Sep 2020 09:17:07 GMT

I was slightly wrong above, we could also solve this problem for Rust by exposing the Buffer class. However, as noted in the previous linked pull request, that requires a unnecessary polyfill for Browsers.

I think the best approach is to just expose TextDecoder/TextEncoder either at global scope or in a require('util') module.

Reply to Allow access to Node's TextDecoder on Sun, 29 Nov 2020 01:18:12 GMT

pruby — Sun, 29 Nov 2020 01:18:12 GMT

As I understand it, Node's Buffer is dangerous to expose from a security perspective. This is because it allows you to allocate and access uninitialised memory... though perhaps in the context of IVM this is cleared? Anyway, at the very least it's a bunch of pointer leaks they probably don't want.

Back to your original idea, providing TextEncoder and TextDecoder would be really useful not just for Rust users, but for anyone wanting to do custom memory or WASM modules. I'm in the JS crowd, but would love to have a proper way to store an ArrayBuffer in RawMemory or segments without an incredibly slow polyfill. JavaScript strings are just awful to deal with.