Writing Shell Scripts with Modern JavaScript

Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON: Hello, everyone, and welcome to another episode of Learn With Jason. Today on the show, we have Mr. Egghead himself, John Lindquist. How are you doing?

JOHN: I'm great. Better that have a hype session. Let's go!

JASON: Yeah, absolute. Let's go! Is Dom watching right now? I'm going to try to make him proud. For those of us who aren't familiar with your work, give us just a little bit of a background.

JOHN: Sure. Development started back on my TI-85 calculator in math class. Instead of paying attention, I was writing games on my calculator. So I failed math but turned into a developer. (Laughter) Which is -- I don't know if that's ironic or poetic.

JASON: I feel like that's the ultimate story. I was real bad at school because I was doing all the things that would eventually make me money.

JOHN: Which is the same story in law school. I dropped out of law school because I was spending too much time reading programming books.

JASON: I had no idea you were in law school.

JOHN: Yeah, so I did that for two years. Or three years. I'm two credits short of a law degree.

JASON: Do you still get crap about that from like your parents?

JOHN: Oh, yeah. All the time. It's great. But the thing is, I walked across the graduation stage, and my picture is up in the hallways, but I just didn't -- there was a summer class that I didn't attend because I already got a development job.

JASON: Oh, okay.

JOHN: So for all intents and purposes, if you Google me in that class at the University of Utah Law School, you'll see my image there. But no, I never took the bar or did anything legal related.

JASON: So you could theoretically, like, run a real-life con like "Suits."

JOHN: Sure, yeah. That's a great show. Is that still on?

JASON: I don't know. I watched like two seasons of it and then it got a little silly, so I stopped.

JOHN: Yeah, about the same. Then development stuff. Then Egghead started after maybe ten years in development. I don't know how time works. Then I just -- with Joel Hooks, we started Egghead seven years ago. Some timeframe like that.

JASON: That's wild.

JOHN: And it started with me and him just making videos and marketing them, and then it's grown into a big old community of wonderful instructors and people who are contributing. It's something that I'm immensely proud of, proud to be a part of, and love with all my heart. It's still boot strapped. Joel and I get to make all the decisions and have complete ownership. If you're one of the instructors on here listens, we love you and thank you for everything you do.

JASON: You know, I'm a huge fan of Egghead. A lot of it is for those reasons. The fact that Egghead is something that -- I mean, and Egghead, the Egghead style is sort of the evolution of your style. You were making these kind of short, punchy videos, and this evolved into a format, which evolved into a platform. You never took venture capital. You never had outside investment. You were just able to grow it into something you controlled and that represented your vision, which is especially in the tech space such a rare and wonderful thing. So it's a big part of why I love the Egghead model, why I'm a big fan of making Egghead courses and all that good stuff. If you're not an Egghead subscriber, it's about to be December. You know what that means. What is it, 12 courses of Egghead? I can't remember what it's called. There's a bunch of December courses that launch. John, did you freeze? Or are you just really -- he's appalled at what I've just said. Oh, no. John, come back. This is going to be a really interesting episode, if I have to write shell scripts by myself. Okay. So while we're waiting for John to come back, which I sincerely hope happens soon, we are going to first screenshot this. Please, everyone, screenshot this because it's wonderful. (Laughter) Then we're going to switch over into the programming mode. Oh, boy. John has now dropped from the call. I assume he's restarting. So everybody look at the empty space where John is supposed to be. Then go follow him on Twitter. You can find John @Johnlindquist. Egghead, the platform, is here. And it is full of all sorts of amazing material, like legitimately the biggest problem that Egghead has is they have so much content that it can be kind of hard to find. So if you want to learn about something, go and Google it. It's going to be really good. We got John coming back. You're back! All right. John, I was a little worried you were going to make me write shell scripts all by myself.

JOHN: All good. I do not know what happened. Internet just kind of went down for a bit.

JASON: We're all good, though. So I was plugging your Twitter. I was plugging Egghead. And I'm going to do a quick plug for the sponsors. So we are being live captioned right now. We have Rachel helping us out today. Thank you, Rachel, for being here. That's available at lwj.dev/live, brought to us by White Coat Captioning. That's made possible through the generous sponsorships of Netlify, Fauna, Sanity, and Auth0, who all kick in to make this show a little more accessible to more people, and we really appreciate that. With that being said, John, what did you want to do today? So we talked -- like, what we said is we were going to do custom shell scripts. That is a huge space. We could do anything with that. So what did you have in mind particularly?

JOHN: Yeah, let me say one thing first. You talked about search in Egghead. If you look at next.egghead.io, which we're launching soon, big old revamped search experience. That should solve -- if you click on the search up there if the top right.

JASON: Nice. We got a filter.

JOHN: A baller search experience.

JASON: I'm very excited for this.

JOHN: So that should -- unless, you know, we're kind of building it live right now. I don't know why.

JASON: It might be in between versions. But that's really, really exciting to see. I'm very, very excited for that.

JOHN: So yeah, what we're doing right now is -- oh, dang. We just got Dee Thompson. He gifted five subs. Welcome to the boop crew. This means you can drown John and I in boops. You can also, using the party Corgi emoji, if you use enough of them, you can turn this into a big old party. Just keep that in mind. That being said, okay. You were going to say something before the fireworks started.

JOHN: Yeah, so shell scripts are one of those things that, I think, everyone wants to be able to do, but if I were to ask you, say, hey, Jason, write a shell script to --

JASON: Wait, I think I just lost your audio. Uh-oh. We lost audio. Having internet problems today. Let's see. Not sure what just happened there. Your video is good.

JOHN: Am I good?

JASON: Oh, you're good. Hey. Yeah, so shell script, if you were to ask me to write one, I would have no idea. Before we talk about the how, let's talk about the why. Like, what does this unlock for us? What does it make possible?

JOHN: Yeah, so that's what I was getting to. I think my audio cut out when I was saying that.

JASON: Oh, sorry.

JOHN: So if I were to ask you to write a shell script to download an image, to reformat it, resize it, and upload it to some other service, I would say, okay, write that shell script and do it in Bash. Like, what would you think? That's impossible. I'm going to have stack overflow.

JASON: Very challenging, yeah.

JOHN: If I with to ask you to do that in JavaScript, what would you think?

JASON: Sounds like what I do all the time.

JOHN: You're like, download two npm packages, some sort of image wrapper or whatever, then I'm done. So that's like the major thing I want to solve. When I have ideas of scripts that I want to run, I want to be able to take that and use those skills I already have and use the entire npm ecosystem that's already available and then combine that into a shell script. So when I started searching that, it wasn't -- like, what most people thought of were CLIs.

JASON: Right.

JOHN: And while setting up a CLI isn't incredibly difficult, it's not the same experience as just opening a file, writing some code, and you're able to run it as a shell script. Because you have to like npm install it globally, set the bin directory, and all sorts of stuff when you're doing a CLI. I thought, this has to be possible, where I can just have a file, right, some JavaScript, and then use that as a script. I mean, JavaScript has script in its name. It's not Java. That's the whole thing. Node runs scripts. So why not use node in the terminal?

JASON: What's up, Joel? Thank you for the sub.

JOHN: So what we can do here is if you -- let's kind of walk you through some of the process. Do you have any other questions about why?

JASON: So I think -- I mean, just to maybe set up a kind of broader view of this. One of the things that I have found myself doing in the past is I have legitimately built apps so that I could solve local problems. I knew how to do this using node, or I knew how to do this using JavaScript. So I would like build something, and then I would stand up a server so that I could run whatever processing job on this server. So what you're saying is that I could skip all those steps, and I could just have that code run locally on my machine without a server, without dealing with express, without all of that stuff. I can just do the bits I need.

JOHN: Yes, yes. Exactly.

JASON: That sounds really, really powerful.

JOHN: Yeah, same thing. A lot of people reach for Electron or I need to write a VS code extension or some sort of binary wrapper, you know, using some tool that helps me do this. But we can just write scripts.

JASON: Absolutely.

JOHN: So and that's -- when you do that, the things you unlock is in the Mac ecosystem, you have Automater or Keyboard Maestro. There's a bunch of programs out there with combined keyboard shortcuts or that can trigger scripts or do all sorts of events that make the scripts happen. So you kind of unlock this entire system of I want something to happen when this other thing happens. Which most people think is, again, you have to write an app for that, which just isn't true. It does require some setup, which could be -- like at the end of this, there can be a pretty fairly simple TLDR of copy and paste this in here, but I would like to explain the steps rather than just giving you something to copy and paste.

JASON: Yeah, yeah. I absolutely want to do that. I think that's the part of this show that I love the most. We usually start with a blank slate and kind of build up from there.

JOHN: Cool, cool, cool.

JASON: So I've got my terminal here. Should I be in my terminal? Should I open a code editor? Where should I begin?

JOHN: Terminal is good. We can start with just create like in your home directory -- looks like that's where you are. Just create a dot JS directory. You can name this whatever.

JASON: Got it. So I'm going to go into dot JS.

JOHN: You could type take.js and that would make the directory.

JASON: What? Hold on. We're going to learn this now.

JOHN: I'm going to teach you everything.

JASON: Yeah, I didn't install any of the fancy stuff. I just switched my shell from bash to ZSH. I think that's what it is because it won't let me do that. All right. We'll have to do another show. I still haven't quite gotten my head around why.

JOHN: All right. So in this directory, just create a file called hello. Just touch a file.

JASON: Yeah, let's open it in code. We'll just open this whole directory. So in here, we've got this empty file. We're going to create something called hello. Now we are in an empty file.

JOHN: And if you try to -- so put like console log hello in there.

JASON: Hello. And thank you for all the cheers, everyone. Really appreciate that.

JOHN: And so our goal from this is to be able to go back into the terminal and to type hello and have it print hello.

JASON: Right. So right now, that's not going to work.

JOHN: Yeah. And do you know why that doesn't work right now?

JASON: I'm assuming right now we have not -- this hello file is not executable, so if we --

JOHN: Correct. So let's do that.

JASON: What is it, al. So we can see this is readable and writable. Then group.

JOHN: Yeah. So if you just want to CH mod that.

JASON: Plus X hello. So CH mod. Mode, is that what it's for? Like change mode?

JOHN: That's how I understand it. The read/write executable.

JASON: So now it's executable. We can see the read/write execute. Still not writable by group or public. For me, I can read it, write it, or execute it. Then if I run hello, still doesn't think it's a command. Do I need to source or anything?

JOHN: You shouldn't have to source. Well, you're in the JS directory, right? Oh, yeah, do dot slash hello. It's trying to run it using the ZSH binary. It's not trying to run it using node. So if you run that with node, just say node hello. Should be fine. It works just fine. So essentially, we're like -- I'd say we're 99% of the way there. It's just that there's always that last 1% to get it to work the way we want. We want it to run by just running hello, and we're going to want a lot of modern JS features to work in there.

JASON: Yeah.

JOHN: So what we can do is just in the file itself at the top line, there's something called a shebang line, which is just the hash and the exclamation point.

JASON: This is actually what the Ricky Martin song is about.

JOHN: Yes. And all the other lyrics that follow that as well. (Laughter) That's all I know. And if you just type node here and hit save.

JASON: I don't have to do a slash or anything?

JOHN: No, because as long as ZSH can find node on the path, and we can talk about that.

JASON: Fascinating, okay. So if I run dot hello, it should run. Oh, dang! Look at that. So then I'm also going to change this to be JavaScript. So that we can read it.

JOHN: And one thing you can do in this directory, since we're just going to drop a bunch of JavaScript files, if you open the workspace settings, I just opened the command palette.

JASON: Oh, yeah. What is it?

JOHN: That one. Top one. So if you search for language and scroll down a bit, there will be one where it matches with the language. Right there. Add item for file association. Just do like star and JavaScript. I think it's JavaScript spelled out. Now you won't have to do that ever again. Instead of defaulting to plain text, it will default everything to JavaScript.

JASON: So this, because it -- yeah, basically we have got VS code settings, now this is going to yell at us. But we don't care. We're going here instead.

JOHN: And yeah, to the point, if -- so for me, these scripts are like my personal shortcuts. These are not something that necessarily put on the server and use as scripts. Although, they could be for sure. And if you wanted to do that, you would use -- instead of node at the top, you'd use USR/bin/env, space node. That way it looks up your environment, finds the proper node.

JASON: And so this is what I expected we were going to have to write. I didn't realize there was a shortcut to just put node in there.

JOHN: And that's why you'll see that everywhere. People want these scripts to be portable between machines. There's probably variations which wouldn't have node. It's all based on your current session and that sort of stuff. So this is safer, per se. And more portable.

JASON: And portability, I think, is probably a good thing to work toward, especially because, like, most of us working in the industry, even if it's not that often we're going to change jobs, get a new machine, get five, six years down the road, replace our laptop. When that happens -- like for example, I have all my dot files and a bunch of scripts saved on GitHub under my dot files repo. And that allows me to just pull this into a new machine, and that plus some stuff that I got from Paul Irish that installed a bunch of apps for me almost one-click configures my entire machine to feel like the same dev environment, which is really, really nice. And this portability, to me -- that gets my brain going like, yeah, we should do it like that.

JOHN: Okay. It won't be as big of an issue in a little while. We'll talk about it in a second. The other thing to point out now is if you go back to your terminal and change directories, like just go up a directory, and you try running hello from here, you can probably guess that's not going to work.

JASON: Right. We'd have to go like dot JS hello. But that sucks. That's not fun. We don't want to have to do that.

JOHN: Yeah. So if you go into your ZSHRC file.

JASON: This will be fun for everyone.

JOHN: And we're stuck. (Laughter) So in ZSH, this file is essentially a file that ZSH will execute each time it opens a new session. So you could write actual code in here, like all those alias show files. All that stuff is stuff you can write in the terminal itself. It's just going to execute all these. And something you can do in ZSH is just path, just lower-case path.

JASON: I think I have this down here. Let's stick this up here, I think. So path plus equals.

JOHN: Then that may have to be no white space. I kind of assume no white space with Bash.

JASON: Sure.

JOHN: Bash is weird. Then tilde, slash, dot JS.

JASON: Yeah, this ZSHRC is actually a copy/pasted Bash RC that I edited until Z shell stopped complaining. So if I showers my Z shell, then I run hello. Beautiful. Look at it go.

JOHN: And a little tip here. You don't have to type source. You can just type dot. Source and dot are the same command. I think source starts looking in directories up. I can't remember the difference. But if you just go dot then tilde, slash ZSHRC, that's the same command. That's the same as source.

JASON: Nice. I did not know that.

JOHN: If you want to save a few characters. If you're as lazy as I am, that's important. All right. So now we have -- now we're lime 99.5% of the way there. We have JavaScript, which can execute anywhere on our machine from a simple script.

JASON: Yep.

JOHN: So the next step I want to show is adding import statements. So right now if you were to try and import FS or something --

JASON: You want to do it like this?

JOHN: Just as you would. We're trying to make a JavaScript system, just as you would, and try and run hello now.

JASON: Okay.

JOHN: And you don't have to source.

JASON: Cannot use outside of a module.

JOHN: Yep. So --

JASON: That should work.

JOHN: I think what may be happening right now is that you showed beforehand your node version was set to 12. Then you set it to 14.

JASON: You know what it is. Node 14 requires dot MJS or your package JSON to be set up with a module.

JOHN: We would be seeing that error if you were on node 14. You're running node from the script. You're not running it from -- the version of node it's finding in the script is version 12.

JASON: Oh, is it really?

JOHN: Because it's not respecting what you set in VM as the default.

JASON: Listen, node, you need to respect my wishes.

JOHN: Yeah, so what you can do here is just say in your terminal, type which node. Just grab that path and drop it in. Or you could --

JASON: Drop this out to there, right?

JOHN: I think so.

JASON: Let's try it. No. Bad interpreter.

JOHN: Oh, yeah. Because it's not loading in any of the environment stuff yet.

JASON: Still doesn't like it though.

JOHN: But now we're getting the right error. This is the error that you were expecting. So what we can do now is if you NPM init in your JS directory, just accept the defaults.

JASON: Don't give me that.

JOHN: NPM init and give it a different name. Sure, sure, sure.

JASON: Yeah, we'll just leave all that. That's fine.

JOHN: Then you know how to set up the module.

JASON: All this stuff we're not going to need. And we're going to set it to module true.

JOHN: I guess in the VS code settings, you could set up that reg X to stop or something. So it's not trying to parse. Remember how we did the workspace settings?

JASON: Yeah.

JOHN: Well, no, I mean limit the JavaScript one so that it doesn't --

JASON: Oh, oh. I see what you mean. So what is that? It's like --

JOHN: Yeah, let's Google.

JASON: How does reg X work? It is going to be --

JOHN: I think it would just be slash -- my mind would jump to the dot asterisk for any character up and to a dot, which would be slash dot, I think. I could be wrong on that.

JASON: Right, okay. So Regex, I always use this tool. Regex Buddy, I think it is.

JOHN: I use Regexr. Something like that. Second link.

JASON: So in here we've got -- we want hello.

JOHN: We don't want to match a dot, yeah.

JASON: And we'll do something else, like index dot HTML. We're going to say we want everything.

JOHN: Yeah, dot asterisk.

JASON: Dot asterisk.

JOHN: And then not -- let's see. What does the not match?

JASON: I think it's like -- so we need this and a not dot. There? That's not it.

JOHN: Yeah, there's a great cheat sheet on the side there if you want to pull it up.

JASON: Where's our cheat sheet?

JOHN: On the left. Third link down. One, two, three. Yep. This is a great tool.

JASON: Negative look ahead. So we need a negative look ahead. So it's going to be --

JOHN: I guess, just does not contain a dot. Isn't there --

JASON: Yeah, does not contain a dot would be this, right?

JOHN: It seems to be matching it. The carrot is the start of the line selector. Everyone -- yeah, the entire group is struggling with Regex.

JASON: That one should work, though. That's the start of the line to the end of the line. Match to the end in JavaScript. Let me escape the dot. I think in this case -- oh, wait. I need to match everything. I think I might bail here so we don't get too lost.

JOHN: Yeah, just delete all that.

JASON: Well, this will do what I want. This is just JSON files and everything else will be JavaScript. That way it doesn't yell at us. Is this the wrong value?

JOHN: I should probably pull up my own.

JASON: Module. What did you say over here? You said it needs to be -- oh, not module but type to module. Okay. So now this should work.

JOHN: Well, this should give you a different error, I think.

JASON: Unknown file extension.

JOHN: So here's where you got ahead. This requires an MJS extension. And this is where I don't want to type MJS extensions. Those don't feel like shell commands to me. I want to use import, but I don't want to have extensions on the end. To me, that's kind of against the point of shell scripts. So what we can do here is if you NPM install ESM, we're going to use the trick in our hello script in the shebang line. If you go to the end of the line and dash R --

JASON: Capital R or lower?

JOHN: Lower. This is just essentially bundling ESM, requiring ESM at a root-level package.

JASON: Interesting. Okay.

JOHN: So you know how a lot of people create an index file or something?

JASON: Okay.

JOHN: Yeah, imagine how a dot env works or something. It's just there. You didn't import something.

JASON: Right. So basically, to recap what just happened, when we run this hello script, the Z shell finds it. It reads this and it says, oh, I should use this binary to execute this script. Then it applies this flag to this call. So what we're actually calling when we type hello is node dash R esm hello.

JOHN: Correct.

JASON: Okay. That makes sense to me. I feel like when that starts to make sense, like when what Bash is doing or what these scripts are doing under the hood, it's not magic. You're giving it a very specific set of instructions. It's just not a very clear programming language. Things just happen, and you just have to know that.

JOHN: Yep, yeah. This file is kind of sending its own script to itself.

JASON: Got it.

JOHN: That's what shebang lines essentially do. Run the code beneath it with the executable at the top configured in such a way.

JASON: Oh. Snova gave us the working Regex. I want to try it just to prove this is possible. Why are you still not working? Not global, multiline. We didn't have multiline on. Some of our other attempts may have worked. This would do what we want. I'm not going to worry about it because we, you know, we're moving on.

JOHN: We're past that.

JASON: But thank you, chat, for teaching me Regex.

JOHN: It can be a subtopic.

JASON: No kidding. All right.

JOHN: So now we're at the place where we have JavaScript running from anywhere using imports and language features on the latest versions, which is a pretty good place to be in.

JASON: I mean, this is really nice. This is a tip that I'm going to take to the bank all over the place. I did not realize I could do that. I've been adding the dot mjs or adding these complicated babble things. So this is wonderful.

JOHN: If you want to have your mind blown, you can do node dash dash help and look at the crazy number of flags -- or sorry, the many flags that are involved there. Because there's a ton of options and ways to configure and run node that we just --

JASON: Like so many. So many.

JOHN: And dash r is in there somewhere.

JASON: So it just preloads, which is really, really cool.

JOHN: And we're going to do that again in a second. So hang on to that thought. So the next thing that we want, this is John's personal take on it. I want all of the things such as top-level await and just having access to read file and write file, all those thing, I don't want to import them in every single file. To me, we're writing scripts, we should be able to write them fast. I shouldn't have to worry about like I'm managing all of it, and I'm not worried about conflicting packages or anything. This is just a bundle of things I want to be able to write quickly.

JASON: Okay.

JOHN: So first, with top-level await, you can try and use it and watch it fail.

JASON: I mean, I'll be honest. I've tried all sorts of different ways of using await, and they all fail. But yeah, top-level await is something that I have yet to write actual top-level await because I've never taken the time to set it up. So what would this be? We can do like --

JOHN: What's a -- let's npm install a package like Axios or something.

JASON: Oh, yeah.

JOHN: Just print some package based on promises. Pick one.

JASON: Node fetch. A nice little baby package.

JOHN: Sure.

JASON: So we can import fetch from node fetch. Then we can await fetch and we'll get, I don't know, just the home page of Google.

JOHN: Yeah, and if you try and run this, it should bum on you saying that await is not possible.

JASON: If I can spell. Yeah, so await is only valid in async function. That's the correct error. So again, we're getting those features. It's aware that await is a thing.

JOHN: Right. So in your package JSON, because we preloaded esm, we can now configure esm with an esm section in there. So under dependencies, if you just drop an esm and an object where you set await to true.

JASON: Really?

JOHN: We now have top-level await in our scripts.

JASON: Wow. Okay.

JOHN: So what you can do is just grab that await and drop it inside of your console log. Since that's top level -- yeah. And that's what I'm talking about --

Holy bucket, did that just work?

JASON: Holy buckets, indeed.

JOHN: When I'm talking about I want to write scripts quickly, that's what I want. I want to console log some fetch request to an URL and print that data. And I don't want to think about it. I just want to do exactly what I know.

JASON: Right.

JOHN: So even from here, like, the step -- I know I'm going to use fetch a lot, right. So we're going to break the cardinal rule of JavaScript and create some globals.

JASON: Okay.

JOHN: Like, this is the one that nobody -- people may strongly disagree with me on this, but I'm writing script, and I want a library of functions that I just have access to.

JASON: I love this. I feel like this is the hallmark of every senior developer, right. Under no circumstances, ever, should you ever do this, but here's why I'm going to do this.

JOHN: Here's the exception that proves the rule.

JASON: Right. (Laughter)

JOHN: Do you want to guess how to do this?

JASON: I mean, I have a theory, which is that I'm just going to stack one of these.

JOHN: Yeah, yep. Exactly.

JASON: Is that right?

JOHN: So the way we're going to do it, because we're going to have quite a few globals to play with, is if you create a -- let's see. Let me double check something before I start saying things. You know how sometimes you just say things and it's not right.

JASON: That is legitimately like the vast majority of what happens when I open my mouth. Like I'm talking and I got to think, okay, where was I trying to go with this?

JOHN: Yeah, exactly. So let's create a globals directory.

JASON: Globals directory, okay.

JOHN: Now, inside of there, let's npm init y.

JASON: Okay. And I want to install node fetch here, I assume.

JOHN: Uh, no. That's not the purpose of this.

JASON: Oh, okay. I'm clearly going in the wrong direction.

JOHN: I mean, it's the right idea. It's just that in here, by default, the package JSON will point to an index file, like an index JS.

JASON: Oh, oh, okay. I see what you're doing. That makes sense.

JOHN: And here we're going to import node fetch. Now, because of the way node works, it should just look up a directory and find its parent packages that we installed.

JASON: Right. Yeah, I think you're right. Then do we want to do it like that?

JOHN: This is not how we roll. That's way too proper.

JASON: Oh, okay. All right.

JOHN: That is too correct. We're going off the rails. Global dot fetch equals fetch.

JASON: What?

JOHN: Yeah, this is how we roll in 2020.

JASON: Oh, dear. Okay.

JOHN: Now go back to your hello, required m globals in that top line. Actually, let's npm install the globals then require them. I think those are the two steps we need.

JASON: Okay. So I'm going back up.

JOHN: Yep. Npm install.

JASON: And that is a tip that I think is worth mentioning. We have installed a local npm package just now, and that's tied to this. So as we change it, this will stay up to date. This is really good if you're trying to build your own npm packages because you don't have to publish them every time to test them out.

JOHN: Yep.

JASON: Just don't forget to use your published version when you finish, or else everything breaks.

JOHN: Now we can preload that sweet little globals package.

JASON: This is -- I'm not even -- this might be -- like, I thought we started in chaotic neutral. I think we're edging toward chaotic evil here, John.

JOHN: I live in chaotic evil. So you can ditch line three. Now we're just console logging and await.

JASON: Good. Oh, man. That feels like it shouldn't work.

JOHN: Right. But it's what I want my workflow to be.

JASON: You bring up a really good point. What we're doing here is not shipping code for other people. I think that's an important distinction to make. I can be really fast if I make a bunch of like shortcut decisions and things like that, but those don't scale because I'll spend so much time explaining why my shortcut works, why I think the way I do to my teammates. But if I'm not working with teammates, if this is just me speeding up my own workflow, shortcut away.

JOHN: Yeah. So let's pass on some arguments now. So if you type -- let's pass the string of Google.com as an argument.

JASON: Okay. We'll just do it like this.

JOHN: So that's the goal, right. In our hello script, we could use process.

JASON: So we can use progress. So we would do like -- just the first one would be like URL?

JOHN: Yeah, I think the first two -- the first one is your node environment. The second one is the current file. Then the third one is the first argument. Yeah, so that's exactly how to do it. Now you can drop URL into the fetch.

JASON: And I'm using underscores to signify we're not using these variables to nothing yells at me.

JOHN: You don't have to use the underscores. They can just be empty.

JASON: Oh, that's right. Yeah, you can just pop them out. I forgot about that. That's also a nice way to do it because it's pretty clear. Nope, nope, yep. It's clear when you get how destructuring works. So I've gotten Google.com. Let's switch that up. My site is so much slower than Google. Geez. But that's cool. So it's taking the argument, and it's switching that out.

JOHN: All right. So my thought process on this one as well, if you grab line four and drop that into your globals index.

JASON: Fascinating, okay. I guess we'll probably just be doing these in a line, right.

JOHN: Yeah, more or less. And do global. I call this one dollar one. Because that's like the standard first argument in Bash and other stuff.

JASON: Okay. So now we have access to dollar one. And we could do something like this and just drop those in, right? Global dot args equals args or something like that.

JOHN: Yes. You could parse those out however you want. You could install some of the command line extensions, npm packages or whatever. You could use any of those. For me, the vast majority of a quick script I write takes the first argument.

JASON: So now this is kind of a Bash standard. If you've ever written a Bash script, this probably looks familiar. If you haven't, you might think we're writing J query. But this is what you'd expect. Dollar sign one, dollar sign two, dollar sign three. So that makes sense to me. So we've set this global. And going back to this, I've saved. Let's run it one more time. There we go. It's doing the thing.

JOHN: Sweet. So now if you go up a directory --

JASON: Up a directory.

JOHN: Yeah. Now try and run it.

JASON: Oh, no!

JOHN: Now, what does it say or make you think?

JASON: That makes me think that because we're running hello, hello gets executed, but then it executes in the current context, which means that it's not finding these packages that were installed in a different directory.

JOHN: Correct, correct. So what we can do here is node has a node path environment variable. So in the -- let's see. In your shebang line at the top, before you invoke node, we can set a node path. So node underscore path and set that equal to the node modules of this path. So where the JS path is. And I don't know if that works.

JASON: This feels wrong. So I'm like copying. It would be like users.

JOHN: Yeah, yep. Slash node modules.

JASON: Then is it just space?

JOHN: Just space, yep.

JASON: Okay.

JOHN: So now with that, node will read in the node path environment variable and know where to look for node modules.

JASON: Bad interpreter. I screwed something up. Oh, quick chat. What's the Z shell equivalent of current working director?

JOHN: PWD?

JASON: That's the one I was thinking of in the first place. Geez.

JOHN: Let's see. Did I typo this? Let's just drop it in. Maybe I typoed. I don't know if you have to escape the dot or something.

JASON: Maybe I can quote it.

JOHN: Yeah.

JASON: Nope. Bad interpreter. Node path -- I don't have to --

JOHN: You set them before and that should -- oh, we might need to use env because I'm using env in mine.

JASON: So, like user bin env dash S. Like that?

JOHN: That may -- yeah, see what happens.

JASON: See what happens. It does it.

JOHN: Thank you, whoever suggested that. All right. So now -- let's see. Are you in the right -- you're in your home directory. So you should be able to test that from anywhere.

JASON: Yeah, let's move into like GitHub.

JOHN: Well, actually, let me stop you before you do that.

JASON: Okay.

JOHN: Just go back to your home directory. I think one thing to be aware of, if you lsla in there --

JASON: My home directory or the JS directory?

JOHN: Home directory, where we ran hello recently. If you scroll up, I think there should be an esm -- let's see. Esm creates a cache, which we usually have to turn off. You don't see like a dot cache?

JASON: I didn't see a dot cache.

JOHN: It's been a while since I set this up.

JASON: Yeah, I don't see one. Doesn't necessarily mean it's not there. There is a -- I don't think that node module is related either.

JOHN: Oh, that might be it. Did it create that today? It did. Yeah, what's in here?

JASON: Cache.

JOHN: Okay, yeah. So instead of littering your computer with node modules caches every time you run a command --

JASON: Got ya, yeah. Probably wise.

JOHN: There's another environment variable we can put next to node path called esm options. Set that to a string, and this is an object, so cache false. No strings inside the object.

JASON: No strings inside the object.

JOHN: Yeah, as far as mine goes.

JASON: Okay. Then that can't go in here?

JOHN: I thought it could. I tried. If someone else finds a way to put that in there, I'd love to see it. But I was unable to. I wasn't sure if that was a bug or something I should file or me misunderstanding the docs.

JASON: What don't you like? Do you just need to not have a space? Is that the problem?

JOHN: Very possible.

JASON: Woof. Esm option mode is invalid. Received cache false.

JOHN: Maybe I'm a different esm version.

JASON: To the docs. Let's go --

JOHN: So Jonathan asked about a huge shebang line. I'm going to talk about that one in a second. We're not ignoring how unwieldy this shebang line is getting.

JASON: I think it's just esm disable cache now.

JOHN: Oh, did they update it?

JASON: Looks like it's changed, yeah.

JOHN: Okay. Well, that's good for me to know. Thank you.

JASON: Dev ops. Where do I put that, though? Can I just put this in here? I wish people would show things in use. That's like my number one pet peeve, when somebody lists something like this and they're not like, okay, what is that, where does that go. Show me a command that uses it.

JOHN: Amen to that.

JASON: So instead, we'll do esm disable cache equals true, I guess. And let's see what happens.

JOHN: May have tried that before.

JASON: It did the thing!

JOHN: You'll have to remove the previous node modules.

JASON: Okay. So remove node modules. There is no node modules. Now I'm going to run this. And it put it back. So that doesn't work.

JOHN: Does that feel like a bug to you?

JASON: This feels like a bug, right? This feels wrong.

JOHN: Okay, all right. Yeah, so I think that should work, and I was -- I use the esm options.

JASON: I don't even see esm options in here anymore. It's easy until it's not, right?

JOHN: And it's always the small things.

JASON: Always the small things. Let's see if someone has asked this question before.

JOHN: Oh, wait a minute.

JASON: See, this is the part -- this is where it gets ugly.

JOHN: Yeah. The idea here is that should -- we are able to turn off that thing where it's creating node modules folders in each. I think reading through this, it looks like we might be able to set a cache to a different directory, which would solve it in a different way. You can try cache zero. I see that in their tests. Like, when in doubt, as a pro tip, if you don't see examples of how to use things, see if people write tests.

JASON: Where did you -- cache test?

JOHN: Yeah, cache tests.

JASON: So they set it to --

JOHN: If you just search for cache or esm options. They have cache one, cache zero.

JASON: Fixture cache path. Esm options. Oh! I think I did this wrong. So I think what I did was esm options. Then they used -- where are your other options? So let's take this. We'll go here.

JOHN: Set it at zero.

JASON: Now you're going to work?

JOHN: Did you type that differently last time?

JASON: Let's just run it again. Doesn't like that. It thinks -- this is a whole different error.

JOHN: Just keep scrolling, just keep scrolling.

JASON: Oh, my god. There's more error than my Bash history allows. That is fantastic. The option mode is invalid, received cache zero.

JOHN: Oh, maybe it's requiring a mode now and it didn't before. So let's see.

JASON: Still looks like it doesn't, though. Am I missing something? I feel like I'm not missing anything. Let's look at the mode. Mode auto?

JOHN: Like, whatever the default is.

JASON: Yeah, whatever, I guess. Sure.

JOHN: Oh, there's an esmrc file in their fixtures.

JASON: Yeah, looked like they had --

JOHN: I wonder if you can do --

JASON: Look, I think that worked. Yep, and there's no node modules. Okay, good enough.

JOHN: Yeah. Yeah, I might use that. Yeah, perfect. Anyway, that was not a requirement before when I did this. So something changed in esm between when I set this up months ago -- I should update esm.

JASON: And we're revealing the kind of cowboy code nature of what's happening here. We're trying to solve problems for ourselves. We're doing it in a way that is a little nonstandard, and as a result, we're at the fringes.

JOHN: I mean, I don't know. It's preloading esm, which is a thing, and configuring it.

JASON: I will say, this should be very -- like, that should be documented. It's wild to me there's not just a do that.

JOHN: So what we're going to do now is in your terminal, create a file in your JS directory. Or in here, yeah. And just call it JS.

JASON: Okay. Now go back to your shebang line. And grab that entire line. And paste it in there.

JASON: Okay.

JOHN: And get rid of the shebang. Now in your hello, you can do shebang JS. This will fail because you're passing an argument. It might try and run. And you need to make JS executable.

JASON: That's true. I do. So let's go to here. And I'm going to go into dot JS, and we're going to chmod plus X on JS. Try that again.

JOHN: Format error.

JASON: Does it need to be relative?

JOHN: Maybe. It's on the path. Try running -- let's see. Format error in hello.

JASON: Format error in hello.

JOHN: Yeah.

JASON: I mean, we can simplify a bit.

JOHN: Remember when I said we were 99% of the way there and it's that last 1%?

JASON: Always the last 1%. So it does not like our formatting somehow. Something we're doing here is incorrect.

JOHN: Yeah, so toss in the user bin env in your hello script. Drop that in front of JS. I think it needs to load in the dot JS path.

JASON: I need to keep it in both places?

JOHN: Yeah, sorry. Keep it in both places.

JASON: Cannot use import statement outside of a module. Oh, it's odd that just now showed its face.

JOHN: I don't think --

JASON: Must use import.

JOHN: I don't think that mattered there. Let's just make sure that's running without imports. Like if you go back to your hello. Let's see. Because it's not loading anything. It's preloading the globals. Then in the globals, we're importing. Okay. And we are configuring the esm options and the package JSON has the type of module set. Let's see. Works on my machine.

JASON: I mean, do I need to make this like one of these? That did something, but it didn't finish.

JOHN: What did you change?

JASON: I just made this into a shebang.

JOHN: Oh, yeah. No, so if you make this a ZSH line -- and sorry. Make a line above that. That's a shebang line. And tell it to run in ZSH mode.

JASON: And for that, it's just bin -- no.

JOHN: I think it's user bins ZSH. Might need a which ZSH. Or the N ZSH will find it.

JASON: Okay. Must use import to load ES module.

JOHN: Are those double quotes messing it up? Yeah, you can try ditching those.

JASON: Double quotes really shouldn't mess this up. Oh, oh. Yeah, that's definitely messing it up. Geez. Okay.

JOHN: So we know it's preloading the globals, and we know that it's defaulting to import and that the requirement is the package JSON. So it's not -- and you're running that from the JS directory?

JASON: Well, yeah, I'm running it from the JS directory.

JOHN: Yep. So there shouldn't be any -- we made this executable.

JASON: We made this executable. So here's our dot JS. That is executable.

JOHN: Chat is telling us to dump the bins-ish. Did we do that already?

JASON: I did, yeah. That's gone. So now we're here. User bin env. We load js.

JOHN: And you understand what we're trying to do there. We're trying to make an executable file that's a version of node.

JASON: Right. I get what we're trying to do. Let's see here. Put the quotes back in node path. I mean, I can try it. But I'm not convinced that's going to do it. I wonder if it's -- what we were doing before, is we were running esm against hello. What we're doing now, I believe, is esm running against globals as well? Like should I put this before? That would be absolutely ridiculous, right? Like, there's no way that's --

JOHN: Yeah, my line has esm before it.

JASON: No way. There's no way that's correct. Because we need globals to be --

JOHN: And this worked -- when we set the mode to auto, maybe that changes the way esm was treating imports.

JASON: Maybe. Let's try a different mode. So esm, we've got all.

JOHN: Looks like it defaults to auto.

JASON: It defaults to auto.

JOHN: Yeah, try all.

JASON: Why not. What have we got to lose? So we'll go all, save here. No. Strict. No. Must use import to load -- this is a weird error, right?

JOHN: It's the one we saw before where we configured type to module in the package JSON. Like, we saw this at the very beginning. And we solved it by -- try ditching -- yeah. We don't configure that in the globals. We configure that in the parent package, which is where we're running things from.

JASON: Right. So now I'm wondering if it's like a -- yeah, it still doesn't like it. This is odd because if we take this back and just put --

JOHN: Yeah, just try it.

JASON: Put this back in here, and it works. So what changes? Something in the way that we're importing this is different. And I don't know what it is or what the difference is. Do you think we need to do something like users, jlengstorf, dot js, dot js.

JOHN: You could try. I think you need to put the --

JASON: Because this one should run it. Oh, but that would be --

JOHN: That's not right.

JASON: So it's not that. Yeah, I don't get it. I don't know what the difference is. I mean, for the sake of moving forward, I'm happy to just copy/paste this back in.

JOHN: We'll just leave it there. I can look into -- oh, that's so weird. It's definitely a works on my machine scenario. Yeah, the concept there is if you create a long shebang line and you want to extract it to its own executable file, you can do that. Then use that in the shebang line.

JASON: I mean, we went to -- like this here, you know, this is a module. And the fact it work when is it's here but not when it's here, that leads me to think there's something bashy going on that we don't know. But whatever. It is what it is, right. We've got this running. It's going what we want. And with our -- I think we've got 15 minutes left here. We can go try to do something fun with this. So we've got the ability now to run arbitrary JavaScript. I think this is really cool because now, you know, like you said, this is kind of a quick setup where if I want to run something all the time, I can just do that. I don't have to go look up stack overflows. Like, here's how you get a list of files when you want to do something complicated with it. I just know, oh, I can install and list that or use fs dot read, which is a thing that I know.

JOHN: Yeah, yeah. Like the entire npm ecosystem is opened up now as far as manipulating images or manipulating files or uploading, downloading, AI. Like anything you want to do that's on npm is now available to us in the shell environment. And where I wanted to get to -- we can try getting there. If you go into -- do you have -- I guess I'll just do it through Automater. This will work differently on whatever machine you have. Do you have the work flow? Did you upgrade to the pro or whatever it is?

JASON: I think so. What's it called?

JOHN: Just open Alfred.

JASON: How do you just open Alfred?

JOHN: Type in Alfred.

JASON: Alfred preferences. Geez. That's actually -- me and Alfred have words sometimes. Just make your own settings accessible.

JOHN: Yeah, so go into workflows and create a blank one.

JASON: As you can see, I'm a heavy user of this feature I got.

JOHN: Blank. Give it a name. And this is -- like, I could spend all day on Alfred workflows. This should be about the same speed as Automater. Just name is show prompt or something. Actually, sorry. Let's do Automater. This will be more accessible to everyone if we do Automater. It's one of those time crunch decisions you make. New document.

JASON: Okay.

JOHN: Then quick action.

JASON: Quick action, got it.

JOHN: Okay. Have you ever made an Automater quick action before?

JASON: Nope.

JOHN: So Automater quick actions are things you can add to right click context menus, keyboard shortcut, all sorts of fun stuff. Across various applications. You can see in the top, workflow receives automatic text to any application. You can set it to chrome, set it to code, set it to whatever. These sorts of things can show up in your right click menus. If you want to bind it to a keyboard shortcut, then you go into your keyboard settings. So just search for shell script. Sorry, your cursor is already there.

JASON: Got it.

JOHN: There's also burn CD in there.

JASON: That's important to me..

JOHN: Yeah, so set that to ZSH. Then we'll invoke our -- so source your ZSH file in here.

JASON: Okay.

JOHN: Then type hello. Well, let's not -- leave off the URL for now. Just the text.

JASON: Okay.

JOHN: Then just in the -- where you search for shell, search for speak text. That should be a good one. And drop that underneath it. Now if you run this in the top right -- is this console logging hello right now?

JASON: No, it's console logging an await.

JOHN: Yeah, just console log.

JASON: That would be a fun speak text. Okay. So quick action, will not repeat.

JOHN: Yeah, it's not getting any input. It tried to do --

JASON: Not found star ship. What?

JOHN: It tried to do the await thing, I think. It ran the error message before.

JASON: That is so big, I can't get to the bottom of it. What happens if you have a small monitor and no big monitor to move that to?

JOHN: Command not found star ship. What is star ship?

JASON: That's this command line prompt, but that should be globally available, which means --

JOHN: Oh, I see. So if that's trying to -- what we can do here, if you have ZSHRC things conflicting --

JASON: That's frustrating because it really shouldn't be.

JOHN: Sometime when is they're run in noninteractive environments, they can blow up on you. So if you just create a dot JSRC --

JASON: A dot JSRC?

JOHN: Sure, just anywhere you can source it from. And move that one line we needed in here. I don't know if we'll also need node path. The idea is in ZSHRC, it sets up a lot of things you dent need when you're going to run scripts from other places from keyboard shortcuts and other sorts of prompts. So you can set up a minimalized RC that -- so it fires faster rather than doing this whole thing.

JASON: So basically what you're saying is now that I've got that --

JOHN: You could source this one instead, yeah. And you can even have your ZSHRC source that file. Why is that blowing up again?

JASON: Who knows. And it doesn't really have a useful error.

JOHN: Yeah, I think what happened is I've had this set up for a few months. When I installed esm, they updated something, and I didn't update esm before.

JASON: That's all right, show. Basically what we're getting toward -- and it sounds like this is maybe a follow-on Egghead course for you, is to kind of show how this would further integrate into your workflow. I know you already have a bunch of courses on this, right?

JOHN: On this thing in particular, no. I've recorded lessons on it. I have 22 follow-up courses on this.

JASON: 22 follow-up courses. Okay. So go follow John for all the information on how that's going to work. At least, Joel says. Yeah, that doesn't surprise me at all. So what I think -- unfortunately, we're kind of running low on time here, but --

JOHN: It was that Regex.

JASON: Yeah, just buried it all. So I'm going to just bail on Automater, but what this shows is that, you know, from a really base level here, what we're able to do is go in and hit a site -- well, I guess we took that part out now. Let's bring that back in. And this is what we'll -- we'll actually publish this so people can check it out. Now we're able to, very quickly, on our machine ping this. If we wanted to go further, we could pull out the headers. So one of the things that I find myself doing a lot is I want to check what the cores errors are. So I need to send an options request. What I have right now is a memorized curl request. I kind of remember how it works. But what I could do is I could use this fetch command and print out just the pieces I need. Any number of things, you know. Processing images or copying something, downloading something from a remote host. What an unlimited thing.

JOHN: So some examples. If you're looking at a URL in Chrome, you can set it up so it grabs the URL from Chrome, and with a keyboard shortcut, you could create a new note in your thoughts project that uses a JavaScript template. JavaScript templates are really easy to write. It pulls that in, sets up a template for you to start taking notes on what you're looking at from a keyboard shortcut. Or it can hit the Twitter API and read some last tweet to you or tell you a joke. All the APIs in the world are available to you. The entire npm ecosystem is available to you. It sucks that we ran into a configuration error of all things because something updated. I personally think it's great. I'm really disappointed we had a configuration thing. I'll figure it out and send an update.

JASON: Like I said, go follow John. Make sure you're following this Twitter account. In addition to just a firehose of really interesting knowledge, John also does music. He's an expert troll. There's good stuff coming on a lot of different things.

JOHN: That's the nicest thing anyone has ever said to me. (Laughter)

JASON: Yeah, I'm here for you. But yeah, so I think from here, we probably don't have time to go any further, but this is a deep rabbit hole. Like, you can go so far with this. I would definitely encourage everybody f you find yourself doing something repetitive, maybe give this flow a shot and see what you can come up with. A lot of our configuration errors are sort of self-inflicted things. Like, we could also have written all of this as hello dot mjs, and we wouldn't need most of this config, right. So there are ways that we could -- you know, it's nice to have the convenience, but there are ways that we can also skip the convenience in favor of speed.

JOHN: True. And I'm also setting up a GitHub repo where you can just check it out and it will be configured for you.

JASON: That would be slick.

JOHN: I wanted to walk through all the steps so that it made sense, what we were doing, and that's where the pain came in. But there will be a GitHub repo. You can check it out. Everything will be configured, and you can just get up and running without any of this hassle.

JASON: Absolutely. Yeah, and so I guess with that, we are going to do a little -- let's do another shout out to the sponsors. We've had Rachel helping us with captioning all day today. She's from White Coat Captioning. Thank you very much. That's made possible by Netlify, Fauna, Sanity, and Auth0. Thank you all for kicking in to make this show more accessible. If you want to learn more, make sure you go over to Egghead. Oh, wait. I have a thing. Check this out. So this is the only thing I use Alfred for, by the way. For snippets.

JOHN: Oh, let me teach you Alfred, please.

JASON: I have an Egghead affiliate code. If you use this code, Joel and John get money, I get a little money, and you get a great education. Everybody wins. Definitely check out Egghead. And I love the tool. If I was not an affiliate, I would still be an instructor. If I was not an instructor, I would be a student. It's a really great tool. Where else should people go to check you out, to learn more, keep up with you?

JOHN: Twitter is the main spot. Egghead, obviously, like you said already. That's pretty much it. Like, I have a blog, which I post randomly to. All my content just funnels into Egghead. I'm a video-first person.

JASON: And you have like hundreds of videos.

JOHN: Yeah. Goal is a thousand by the end of the year.

JASON: Dang. How close are you?

JOHN: That's a joke. I'm at like 550. I don't know. I should check.

JASON: You can totally do that.

JOHN: I can get there.

JASON: Six a day.

JOHN: Oh, yeah. You can follow my sound cloud at Mr. Lindquist, if you want. There you go.

JASON: Okay. Sound Cloud, Mr. Lindquist.

JASON: Yeah, I think one of the songs I wrote about you is on there.

JASON: Yeah, I'm so into it. This is one of your really good trolling things. You'll pull samples. Like is this -- you've trolled a handful of people this way.

JOHN: You can scroll up to you freaking rock. Is that Dom?

JASON: Yeah, but where is it? There it is. (Music) I may actually route this. Start it over. (Music)

You freakin' rock, man.

JASON: I love it. It's so much fun. Y'all are great for hanging out with us. John, thank you so, so much for hanging out with us today. I don't even know what to say. I had a blast. I know this was maybe not as much fun as you were hoping to have, but I still had a whole lot of fun.

JOHN: Yeah, so much potential.

JASON: It's all right. We can always do a follow-up. Chat, make sure you go and check out the schedule. We've got some fun stuff. We're not coming back on Thursday because Thursday is Thanksgiving in the United States. So I'll be taking that day off. But I will be on the free code cam stream on Friday with Kurt. Then we're coming back, and we're going to learn Redwood JS on Tuesday. Make sure you check out the schedule. You can add this to your calendar. It will be a whole lot of fun. Stay tuned because we're going to raid. John, thank you one more time for hanging out with us.

JOHN: Thank you, Jason. You're awesome.

JASON: See you all next time.