So for those unaware, I’ve been spending a fairly long time on typescript integration in TMT-X. In the devlog I made earlier this week I said I was looking forward to wrapping it up for now and moving on, but even having what I thought were the last big issues resolved, I’ve continued to find new ones that prevent this perfect createLayer function from materializing.
I’ve been thinking and experimenting with various other methods of describing layers that would give this type safety in a more reasonable amount of time. While I’d like to keep the API here as close to TMT as I can, I decided I’d explore one of the more promising alternatives here, to get a sense of people’s thoughts on it. This is fairly technical so I put it in #general:modders-only.
The basic idea is, instead of creating one massive object, you create each of the elements separately and then compose them together. For example, this very small layer:
const foo = makeUpgrade({
cost: 4,
description: "foo",
effect() {
if (bar.bought) {
return 10;
}
return 5;
}
});
const bar = makeUpgrade({
cost() {
if (infinity.upgrades.upg1.bought) {
return 1;
}
return 2;
},
description: "bar",
unlocked() { return foo.bought; },
onPurchase() {
console.log("blah", this.cost);
}
});
const prestigeEffect = computed(() => {
let mult = 0;
if (foo.bought) {
mult += foo.effect;
}
if (bar.bought) {
mult *= 2;
}
return mult;
});
export default makeLayer({
id: "prestige",
effect: prestigeEffect,
upgrades: { foo, bar },
onReset() {
console.log(this);
}
});
Now lets talk about some of the implications of such a large change.
Typings
So first off, one thing this does really well is typing. You’ll notice there is not a single type annotation anywhere in the above example. Despite that, everything is able to be type checked appropriately, and in fact typescript is smart enough to even tell you things like that bar.cost will always be either 1 or 2, rather than defaulting to “some number”.
This is true even for references to stuff on other layers, and other parts on the same layer - even if they’re defined afterwards.
What this means for the creator is your IDE will have much more useful auto-complete suggestions, and can catch type-related bugs for you without you needing to test anything. I feel that’ll cut down a lot of debugging time for basically everyone, which is fantastic.
Accessing this layer
Something I was playing around with for awhile is making this.layer and this.player always access the layer, or that layer’s corresponding object in player, respectively, when accessed from anywhere within a layer. In addition to making the typing much harder to figure out, I didn’t really like the non-intuitiveness that this.player would be equivalent to player[this.layer.id]. And ultimately the goal of this.layer was to fix the issue that currently in TMT and TMT-X its nearly as hard to access the current layer’s features as it is to access a completely different layer’s features. Indeed, you’d usually end up just using the utility functions with this.layer for the layer ID.
With this new syntax, you can just access the parts of the layer directly. They’re already in that module’s scope. e.g. to get the upgrade called foo you don’t need to do getUpgrade("prestige", "foo") or layers.p.upgrades.foo or even this.layer.upgrades.foo. It’s just foo. Literally couldn’t get any easier than that.
Modifying the this variable to add these layer and player properties had the unavoidable consequence of stripping away the normal properties on this. The new syntax avoids that - instead of replacing the existing properties with the 2 new ones, instead it just modifies the this property to be the proxied version of itself, so the type is actually properly typed on every item.
Note there is one caveat. While the makeUpgrade, makeLayer, and other makeX functions will all return a proxied object, if you define a function by wrapping it in computed as shown in the example above, it won’t actually be a part of a proxy yet. That means to get the cached value of the prestigeEffect function above, you’d have to type either prestigeEffect.value. Alternatively, if instead of directly exporting the result of makeLayer you stored it in a object and then exported it, you could use that object elsewhere in the layer.
e.g. if you do
const prestige = makeLayer({
effect: prestigeEffect
});
export default prestige;
then the rest of the layer could access prestige.effect.
Accessing other layers
So as you’re probably aware, since TMT-X uses a bundler, its required to import things when using them. Here’s roughly what the import statements would look like for the prestige layer example from above:
import { makeUpgrade } from "@/features/upgrades";
import { makeLayer } from "@/game/layer";
import infinity from "@/data/layers/infinity";
Fairly simple, although for some of you that last import statement might be looking very problematic. You may be worried about a cyclical dependency, if the infinity layer needs to import the prestige layer. Which, indeed, it would:
import { makeUpgrade } from "@/features/upgrades";
import { makeLayer } from "@/game/layers";
import prestige from "@/data/layers/prestige";
const upg1 = makeUpgrade({
description: "upg1",
cost: 1,
unlocked() {
return prestige.upgrades.bar.bought;
},
effect() {
return prestige.effect;
}
});
const infinity = makeLayer({
id: "infinity",
upgrades: { upg1 }
});
Fortunately, this actually works just fine! While cyclical dependencies can be considered a bit of a code smell, in this case the way layers are structured you should never actually access another layer in the top level - only within various functions that don’t get resolved until the layer is actually loaded.
Realistically, the only alternative to importing layers like this, would be importing the type as well as a global layers object that wouldn’t actually know the types of any specific layer. Something like this:
...
import { layers } from "@/game/layers";
import type prestige from "@/data/layers/prestige";
const upg1 = makeUpgrade({
description: "upg1",
cost: 1,
unlocked() {
return (layers.p as typeof prestige).upgrades.bar.bought;
}
});
...
This is more verbose, and offers less safety, since it can’t check if layers.p is actually typeof prestige. If you think the various utility functions may help, this is what they’d have to look like:
hasUpgrade<typeof prestige, "bar">("p", "bar")
They too are longer than the TMT equivalent, but even worse it now needs redundant information within the type parameters and the function parameters themselves. And it still is just assuming at runtime layers.p will actually be typeof prestige!
I think a bit of cyclical dependencies, that realistically will never actually cause an issue with how this framework is setup, is preferable. And that actually goes for any syntax for defining layers, not just this one.
Feature agnosticism
One of the tricky things I’ve had to deal with, when creating this massive createLayer function, is dealing with getting correct types of a bunch of objects that are all grouped by type. That is, prestige.upgrades is filled with Upgrades, but each Upgrade may have its own unique properties, or return different types within its effect property, etc.
This new syntax, as currently implemented, doesn’t actually say “the upgrades property should contain a bunch of string - upgrade pairs” - in fact, the upgrades property isn’t actually defined at all! the makeLayer function just defines a couple absolutely necessary properties like id, and says it can have any number of additional properties, all typed as unknown. The return value of the makeLayer function retains those properties, although they’ll now infer the types from the object passed in as a parameter.
What this means is that you can do prestige.upgrades.foo and it’ll give you exactly any properties that exist on that specific upgrade. Additionally, if you tried to do prestige.upgrades.invalid the IDE would give an error about invalid not being a property of prestige.upgrades.
That probably already sounds good, but maybe more relevant to the “typings” section above. The reason this is in its own section is because not defining the feature properties on a layer manually means its trivial to add in new feature types. Someone can pass around a firstPerson feature (just for example) all wrapped up nicely in its own file, with a makeFirstPerson function you can call and just include in your layer, and it’ll all work just as easily as the existing features. This could heavily encourage people to create custom feature types and share them with others, because it’ll be so easy to do, and you don’t need to worry about merging all these custom fields into the player type definition file.
Feature variants
Another benefit of this isn’t just custom features, but being able to create different versions of a feature - because really there won’t necessarily be feature types. Just various objects inside a layer. That means instead of just having a makeUpgrade function, a feature might have a makeDiscoverableUpgrade function that is like 95% equivalent to the current upgrade feature, but with some extra bits that make them behave a little differently. Indeed, you could see how something like a buyable is really just a clickable variant, slightly modified to be more easy to use for a common use case. This new syntax would encourage more of these variants, and could in theory include them without cluttering up the layer object: you could just stick the discoverable upgrades in the same upgrades property.
You may have noticed I’ve also chosen an incredibly easy example feature this whole time. Upgrades are easily the most basic feature type in TMT or TMT-X. What about something like buyables, which can have that resec button and various properties tied to that? Also what if a player wants to have different sets of buyables within one layer that each can be respec’ed independently (something not possible in TMT either, btw)? How would that work with these functions that only make single items at a time?
Well, I think the solution is something like a makeRespecButton() function, and various equivalents for the other “complex” features, that asks for the relevant information. This would also solve the issue of microtab families. In fact, I think there would probably be a generic makeTab function, a makeTabFamily function, and layer.display can be set to a tab or a tab family. And then ofc within a tab’s display you could embed other tab families. Ultimately, the distinction between subtabs and microtabs was incredibly arbitrary and I think this would make a lot more sense.
BTW, with typescript I wasn’t able to nicely handle having these kind of “feature type”-scoped properties in the same object as the features themselves, so in the old syntax it made you do things like layer.upgrades.data.foo instead of layer.upgrades.foo. This syntax would get rid of that issue, and in a much cleaner way that allows you to have multiple “feature groups” of the same feature type.
Structure agnosticism
Alright, this is a big one, that I’d argue a lot of these other points have naturally pointing towards as the natural conclusion of this syntax change. Let’s say we did in fact make it so the upgrades layer is no longer defined as a map of IDs to upgrades, and continue modifying things so it makes fewer and fewer assumptions about what an upgrade looks like, pushing more and more of the logic into the makeUpgrade function. At a certain point it wouldn’t actually matter if the upgrade is in layer.upgrades… so why keep it there?
Having it somewhere in the layer is useful for various reasons like organization, but what if we just let the creator decide their own structure for any given layer? I don’t think we’d necessarily want to encourage, let alone enforce, that things are organized by feature type. If there’s one main clickable in a layer that triples the point gain, for example, why couldn’t it just be stored at prestige.tripler? Or let’s say the prestige layer had various subtabs, and we wanted to organize the features within the specific subtab, e.g. prestige.main.upgrades vs prestige.modifiers.upgrades.
Not only would this give the creator more control and flexibility, but it would further encourage mods and features to be developed with modularity in mind.
I’m a little concerned about the fact that this might make it tricky to access all features of a given type, but I’m not positive that actually makes sense. The current go-to answer would be how to display “all upgrades”, but tbh I think that’s a bit of a mess anyways. It’s currently relying heavily on assumptions that all upgrades should be shown at once, and that the upgrade IDs determine where it’s placed relative to others. I like using descriptive names for the upgrades, which is incompatible with that. We can find possible utility methods to make it easier to create template strings for displaying groups of upgrades, but even without that I think it’s a lot more manageable to type something like
<upgrades :upgrades="['doubler', 'tripler', quadrupler']" />
than the current TMT way:
["row", [["upgrade", "doubler"], ["upgrade", "tripler"], ["upgrade", "quadrupler"]]]
And of course there would definitely be a utility function that would take an object of upgrades with numeric IDs, and create the usual grid layout used in TMT.
Common code / inter-feature communication
There are pieces of code that currently assume only specific features exist. For example, the default layer display would not have a section within it to display any firstPerson features (or any other custom feature). That means using these would require you to write your own display template. To be honest, I want to encourage creators to do that anyways, but it’s something to keep in mind.
Some features also have code that runs every tick. For example, achievements and milestones check their requirements. To be truly feature agnostic we’d need some way for these features to register a function to run every tick instead.
And some features are highly coupled with each other. For example, resets are deeply tied to basically everything in the engine. Most notably, the challenge and tree features. What if a new feature needs to also tie itself within this nest of highly coupled features? That’s not very easy to do without causing merge conflicts for anyone wanting to use multiple custom features, and in general high coupling should be avoided anyways.
The main solution to this would be some sort of event bus. They’re fairly straight forward to implement, but will be a further deviation from how things work in TMT. Fortunately, this should mostly just affect people interested in making custom features - and making it easier to make and share them. For creators who just want to use features - built in ones or custom ones - this should only make things easier.
With this implementation, you could also completely avoid some of the issues with the structure agnosticism mentioned earlier: the makeX functions could register itself in the proper places and then it doesn’t matter where in the layer object the feature actually resides.
However, there’s a ton of edge cases that an event bus won’t solve, like how to construct what a default layer display would look like.
Accessing save data
Save data will be pretty tricky. I actually don’t have a good way of making this type safe easily, which is probably my biggest concern with this new syntax. First off: When should the location of a new piece of save data be decided? If I create an upgrade called “foo” should it make a boolean at layer.upgrades.foo? Well, that wouldn’t work because the upgrades don’t actually use IDs with this syntax. I suppose the closest equivalent is where they are in the final composed layer. That’s an elegant solution, mirroring the structure of the layers and the player.
But… how will the items know where they’ve been placed? When you call makeUpgrade it won’t know where the upgrade will be placed into the layer, so it won’t be able to create the save data at that point. Maybe it’ll just not create a .bought property, or .state, or .earned, or whatever other function usually reads from the save data on player, but assume it’ll be defined when the makeLayer function is called. But that isn’t type safe, and how would the makeLayer know the name of the property that should look into the player save data, and how would it know what type of data should be stored there, and its default value? And what if the item is accidentally left out of the layer object! If another item accesses it directly then it’ll just error about bought (or whichever function it is) being undefined!
I really want a better solution, but my current idea is to require a getter and setter for each of the makeX functions that will be used to save/load from the player save data. But really, that is massively inconvenient, even with some utility functions to help out.
Oh, btw I’ve come to really dislike how we create player.subtabs[layer].mainTabs for storing which subtab is active in a layer, and I’d love for that to just follow the same structure of the rest of the layer.
Default feature values
There’s one final issue, which I’d like to introduce through an example. In the prestige layer described at the top of this post, the foo upgrade has a cost of 4. 4 what? 4 points? 4 layer points? Similar to the accessing save data issue, the upgrade doesn’t know which layer it’s been added to, so there can’t really be a default function like there is in TMT, that just checks layer.points unless otherwise specified.
And really a lot of features have default values that reference the layer it’s on. I guess you could solve this by asking for a layer id in the makeX functions, but that feels a bit code smelly. Plus you wouldn’t get any type safety this way: I can’t let the makeX functions take in the actual layer object either, to check the property actually exists, since you’d be calling makeX before the layer object is created.
Technically that solution would still work, but I’d really like it to be type safe and… not smelly.
Conclusion
This is a very large change, and one that I can see as being pretty controversial. It’s not a perfect syntax by any means, with a couple unsolved problems (most notably not having a good way to have a default layer display, and how to structure save data), and I don’t want to move away from the TMT syntax for no reason - it just makes it harder to convert TMT projects into TMT-X projects. However, I’ve been spending a lot of time trying to make this createLayer function work, and I really want to move on. Plus I think this method could be really significant in making the engine as a whole more modular, which I think is a really good thing.
Anyways, tell me what you think. For now I’m going to move my latest attempt at making a createLayer into a different branch and keep working on other TMT-X features I need to work on instead.