Alright I’m going to verbally note down some very interesting thoughts that we had just with Shanni right now.
So we were just talking about music, we got into the topic of music and I said it’s interesting that it’s been so long since I played the guitar and then we conversed a bit more and it’s like actually the Launchpad, even that I’ve only used very very sporadically here and there like three three four times total probably in the last two years maybe a year and a half but um then i realized or then she says or well i don’t know one of us says and we start discussing about the idea of using or she’s i think you you told me right like she tells me like use the you should use it to do something
launchpad
I forget how I started it’s okay we can fill it in later but the point is we end up on the topic of using the launchpad as an input device and like what that would look like and initially like I was thinking in the sense of like you know the 8x8 grid and you only could I was thinking about it using it as a direct like Controller for NeoVim like very focused very precise to focus to precise to precisely focused on that single domain And I was thinking of like I guess you could like represent on the pixels You could represent like a mini-map on it, and you could like touch to navigate your code That’s that was the only thing I thought of and then she’s like we maybe to go back and forth a little bit on that and then she’s like But what if you used it to do more or something?
expanded idea
Again, we can fill that in later.
But she says you can use it to do more things.
Fast forward, we end up discussing… We end up on the topic of using the way that I already know how to play it in a unique way as opposed to just a simple… you know press this to do that press like a basically a glorified steam not steam deck stream deck or whatever those are called the Elgato stream deck like literally just buttons right that you just map to actions like it’s nothing it’s it’s totally like you could do the same thing with your keyboard there’s no point in doing that but chords and not rhythms but chords like are actually very interesting.
For the same reason that i3 layers are very interesting, i3 modes, I should say. Or like, what’s this other keybinding system for Linux? It’s not just for Linux. It’s for Linux, Mac, and Windows, I believe. Or maybe it’s just Linux and Mac. But it’s like this very, very rich keybinding system that is favored by people who have that gigantic keyboard that costs a lot and is very good. It’s a competitor to the Gleb80. and they use it to, or I found one configuration that was somebody using it to do, they had like five or six layers and you can do theoretically any number of layers with it.
chord interface
That is basically the kind of system… In fact, we could just make this a keyboard in that system and then you… I don’t know, maybe not. I mean, really, you can do both. You can have, like, one layer of MIDI be translated into the keyboard and then you can have other MIDI channels that are simply MIDI, right? And then you can have some listener that listens for the key presses on that layer or who knows, right?
The point is, though, I was thinking about, from that point, like what if you Like, in the way that the Caricorder feels to observe, as someone who doesn’t use it, this system would feel similar to Observe, because it’s rich, it’s dense, it’s deep. It would be a system that you can effectively use to navigate your computer in one way or another, and I think I would be very closely mapping it to window managers, specifically i3, Sway, and… Wow, I just had a really interesting thought. But yeah, specifically i3 Sway and the Mac equivalent, this aerospace one.
That would be probably like step number one is just build a system that lets you use cords, some set of cords. to navigate your windows to like move them to reposition them to resize them to switch desktops like basically everything that you can do in with those key bindings not necessarily everything some simplified set but a focus set that is enough that you are equally proficient as somebody on a mac at least at moving around the windows not necessarily typing not necessarily moving the mouse and interacting with anything within the windows but at least moving around and repositioning the windows and defining layouts like quickly assembling layouts just how you want them like almost like if you’re sitting at a table and like I’m gonna move this stuff around just how I want it like it’s very simple it should feel at least as effective as that if not more effective where currently most computers even with keyboards you’re not that effective at it like it’s compared to a mouse a keyboard layout with a window manager is really really powerful
comparison
but unless you’re like I guess really good at working with the window managers or you have an awesome window manager lets you do really cool things that’s a very interesting design you’re not going to be like telepathically moving things into place I mean it’s all relative you basically are but what I’m saying here is like it really is like it’s a relative leap it’s like an order of magnitude greater of a human-computer connection achieved by using cords as interface or something like this where you enter and exit like I was telling Shanni like similar to how the iPhone and the touchscreen like a touchscreen on its own like in 2002 or whatever touchscreens existed back then maybe at least 2006 they existed right but they were very very very simple they were resistive they were single point The interfaces were very bad, right? Like, I remember the cafeteria lunch ladies had these buttons they would press, and it’s just like big red buttons, same as the LaunchPad, that’s what I’m describing here. It’s like, okay, click, click, click, click, click, click, click. It’s like a mouse would be not much worse.
future potential
So… And then here we are, you know, well even just a few years after that, just two years after that, the iPhone comes out and they totally redefine the way that it feels to interface with a touchscreen. Like they build really interesting UI components. They don’t have Swift yet, it’s Objective-C, but eventually… Fast forward more, here we are. We can very quickly assemble applications with predefined components from Apple that really, really feel solid and very clean to use. You can define new and interesting ways to interface with things, or at least to represent things, given their abstractions.
Yeah, this is basically that. This is to that. Or this is to a launchpad what iOS or rather like Swift AppKit components are to touchscreens and Really, Launchpads could be a whole class of applications. Like, what is this tracker? I wonder if it does actually produce… There’s like the PolyTracker or whatever. Wait, did we already pay? Oh, okay. The PolyTracker, I believe it functions with… What I’m saying, I wonder if it functions in that you can read all the MIDI in and out. Like, is there a GitHub repo for the PolyTracker? Or whatever that thing is called. PolyEndTracker.
chord-based interface
Anyway, so… yeah just imagine a chordal interface that lets you not music well actually you can use music to navigate I was telling her like what if you like a go dun-dun-dun dun-dun-dun like what if you play that and you it’s not dun-dun-dun it’s like chords it’s like chord progressions like there are certain chord progressions that are very easy to play on the launchpad or on a guitar easy to transition between easy to flow between easy to use to express something interesting but rather than expressing something interesting in a musical sense we’re just using those musical like grammars to Neofim
Yeah, I think Neo is actually a really good way to look at it as well. Like maybe we can make this modal also, like you have certain chords that trigger certain modes, and you have certain chords that leave certain modes, or maybe that immediately jump between modes, kind of like Zelig, right?
so imagine like if you play like a certain it’s kind of like in some songs like you might have like a very like home note that you’re like keep like playing here and there and maybe that’s like your escape key right and you press escape you go back out to your main root mode like mode as in modal but then maybe you also have some other like a high point of that note right so you have like a dung and then the three octaves up three octaves up you have a dun right and you dun dun dun dun or like you know three octaves up and in the middle you have your chords and your chords are your way or your interface and this is all just one possible way to do it it could be simpler than that it can be less musical than that um it’s just a matter of like chords as input method
conclusion
That’s it. I think this is a really interesting research direction. Like, genuinely. Especially as we enter the realm of language models. Oh man, we’re gonna get soaked.