Goin’ back to Windows: multiple terminal windows with ConEmu

This is part of a series on moving from desktop Linux back to Windows.

The first post is here. The previous post is here.

Back when I was a full-time software developer, working on a Windows machine, I rarely needed cmd. I’d write batch files, sure, but I could launch those with Launchy or AutoHotKey or a toolbar mouse click.  Having a cmd window open all day just wasn’t a thing, for me. The only thing I might need a shell for was subversion or git, but most likely I used file system integration (i.e. point/click… boo, I know) or whatever IDE I was using at the time.

When I switched to Mac, and eventually Linux, having a shell running all day long was just how things worked. Right now, on my work laptop (MacBook Pro), I have half a dozen+ iTerm2 tabs open.

When you have a powerfull shell, you use it, a lot.

Multiple terminals on Windows

WSL made a powerful Linux shell on Windows a reality. But as of this writing, opening an Ubuntu bash shell only supports a single window. Sure, you can open multiple, separate shells, but that’s like web browsing pre-tab-browser. No thanks.

ConEmu makes it as simple as iTerm2 or Terminal to have multiple shells on Windows. It’s even easy to have multiple different terminals within a single ConEmu window. In my experience, the combination of WSL, supplemented with ConEmu, has made Windows finally stopped feeling like a second-class citizen development environment.

Check it out:

Just like Terminal (Linux) or iTerm2 (Mac), you can use the keyboard to create new tabs, cycle through tabs, and the like. apt install tmux and you can tmux, too.

ConEmu is highly customizable, though I tend to keep things default and just add keyboard shortcuts. My current setup is that an Ubuntu bash shell is the default shell, activated by the default win-w, and I’ve assigned win-p to Powershell. Here’s how to do that:

Since I’ve moved back to Windows, the combination of WSL for a Linux experience, and ConEmu for managing multiple terminal windows, has been a delight.

If you use Chocolatey, install it with choco install conemu, and off you go. Otherwise, download it at https://conemu.github.io/

Goin’ back to Windows: Launchy

This is part of a series on moving from desktop Linux back to Windows.

The first post is here. The previous post is here.

A very long time ago, before I had ever used Mac or Linux for personal computing, someone had convinced me of the value of a “launcher”: a program, usually invoked via alt-space, that would pop up a box and help you find stuff on your computer, launch programs / scripts, do quickie things like calculations, and otherwise keep your hands on the keyboard and off of the mouse.

At that time, the only game in town for Windows was Launchy.

When I started using a Mac for work, I tried out Spotlight, which is the default Mac launcher, and it felt OK but not even on par with Launchy. I quickly discovered Quicksilver and have stuck with it.

When I moved to Linux a few years ago, I started using Kupfer, though I don’t recall why. It worked just fine, but I was a n00b and had I known about GNOME-Do, I probably would have used that.

When I moved back to Windows, one of the first things I did was look for the current state of launchers on Windows. And, to my surprise, it seems that Launchy is still a favorite. Here’s what it looks like, exactly the same as it did in 2009:

Why not just the win key?

The win key is fine as an application launcher. It’s easy, fast, and just works.

What I like about Launchy, though, is that it also makes it easier to navigate the file system quickly. For example, let’s say I keep all my code in c:\dev\projects. If I want to navigate to that natively, I could hit the win key and then type c:\dev\projects. Or I could open up explorer and point-and-click to it.

But with Launchy, it’s as easy as

This is possible because Launchy lets you configure where it looks for stuff. In the case above, I can configure launch to catalog files or folders in a certain location:

Finally, Launchy includes a catalog of plugins and comes with some useful defaults. For example, I often need to so simple-ish calculations, and Launchy makes that really easy thanks to the Calc plugin:

Conclusion

This is all certainly not life-changing, earth-shattering stuff. But I spend a lot of time on a computer, and pointing-and-clicking all day long is inefficient and unenjoyable. I like tiny time-saving, joy-boosting things, and a launcher like Launchy serves nicely.

Lambda: using AWS SAM Local to test Lambda functions locally on Windows

This is part of a series on moving from desktop Linux back to Windows.

The first post is here. The previous post is here.

AWS SAM Local is a “CLI tool for local development and testing of Serverless applications.” It uses Docker to simulate a Lambda-like experience. The docs explain well how to get started, and the GitHub repo has lots of samples as well. As of this writing, it supports python, java, .net, and nodejs.

This is a quick post to show how to use it in Windows Subsystem for Linux (WSL) and Docker For Windows.

Installing on Windows

The instructions recommend installing with npm. That didn’t work for me, giving me errors about file not found. I’m not sure if this is a problem with npm inside of WSL, if it’s a problem with the current installer, or what.

I did get it installed by using the next option, which is go get github.com/awslabs/aws-sam-local. I then aliased that to sam: alias sam='aws-sam-local'

Running on Windows

Out of the box, using Docker for Windows as the Docker daemon, the sam command itself worked fine, but actually invoking a function did not work for me when using the docker client on Ubuntu within WSL. With a simple python hello-world example, I’d get this:

marc@ulysses:sam-local-play$ sam local invoke "HelloWorldFunction" -e event.json
2018/01/20 08:59:18 Successfully parsed template.yml
2018/01/20 08:59:18 Connected to Docker 1.35
...
Unable to import module 'main': No module named 'main'
END RequestId: c9edd13a-000e-49bd-a4a7-a8a23258a03b
REPORT RequestId: c9edd13a-000e-49bd-a4a7-a8a23258a03b Duration: 1 ms Billed Duration: 0 ms Memory Size: 0 MB Max Memory Used: 19 MB
{"errorMessage": "Unable to import module 'main'"}

In the instructions below, I’ll tie together several GitHub issues and a gist from three separate GH users.

First, Kivol, in this  GH issue comment from GH user, shows a 2-fold solution:

1. bind-mount /c to /mnt/c and then within Ubuntu make sure you’re on /c (from this comment on another GH issue from aseering)

2. pass --docker-volume-basedir in the sam invoke command

Here’s how to do all that (again, mostly copying from a few GH issues and adding some color commentary):

# bind-mount /c. this mount lasts as long as your current terminal session; instructions below for how to make this persistent if this works for you
$ sudo mkdir /c
$ sudo mount --bind /mnt/c /c

# now you can use /c instead of /mnt/c
$ cd /c/path/to/project $ sam local invoke --docker-volume-basedir $(pwd -P) --event event.json "HelloWorldFunction"

And voila, it worked. I get this beautiful output:

2018/01/20 12:16:08 Successfully parsed template.yml
2018/01/20 12:16:08 Connected to Docker 1.35
2018/01/20 12:16:08 Fetching lambci/lambda:python3.6 image for python3.6 runtime...
python3.6: Pulling from lambci/lambda
Digest: sha256:0682e157b34e18cf182b2aaffb501971c7a0c08c785f337629122b7de34e3945
Status: Image is up to date for lambci/lambda:python3.6
2018/01/20 12:16:09 Invoking main.handler (python3.6)
2018/01/20 12:16:09 Mounting /c/dev/projects/lambda-projects/sam-local-play as /var/task:ro inside runtime container
START RequestId: d7a9dcb0-751e-445d-9566-a025f4e804b0 Version: $LATEST
Loading function
value1 = value1
value2 = value2
value3 = value3
END RequestId: d7a9dcb0-751e-445d-9566-a025f4e804b0
REPORT RequestId: d7a9dcb0-751e-445d-9566-a025f4e804b0 Duration: 4 ms Billed Duration: 0 ms Memory Size: 0 MB Max Memory Used: 19 MB

Note: that cd /c/... is really important! If you do all the above but stay in your shell at /mnt/c — like I did 🙂 — it’s still not going to work.

Big thanks to GitHub users Kivol and aseering for putting this together!

Persistently mounting /c

If the above worked for you, then you’ll probably also want to persistently mount /c so that you don’t have to redo it every time you want to use sam from within Linux / WSL.

If you’ve got a Linux background, you’re thinking: just mount it in /etc/fstab. It seems that as of now, anyway, WSL isn’t loading /etc/fstab entries when you open a new shell (i.e. it seems as if you have to run mount -a every time), at least according to comments in this MSDN post and this WSL issue.

Fortunately, linked in those comments, sgtoj has a gist that sets all this up nicely. I saved this locally, ran it once, and now /c is mounted for all new Linux sessions.

Really?

Coming from doing all personal development on Linux for the past 3 or so years, these kinds of hacks are disappointing. So far in this experiment with going back to Windows, these hacks have been few, and so far for me have all been related to wanting to use a docker client inside of Linux / WSL. More on that in a future post.

Suffice to say: yeah, it’s hacky, but it’s not that bad. Annoying, sure, but certainly not enough to sully the overall experience so far in moving back to Windows. I can live with this one.

A note on aws-sam-local and Powershell

After initially failing to get aws-sam-local running successfully within WSL, I figured I’d try it out all on the Windows side of the house. I ran into problems there, too. First, using go-get to install it, I got path-too-long errors. WTF. That led me to choco install nodejs and use the npm install route. That did work, and sam invoke worked fine.

So far, this is the first time that working in WSL was kind of a shit-show. Thankfully smart people figured out how to get it working correctly. I am always grateful when I find answers in GitHub comments along the lines of “maybe sharing this will help.”

I really did not want to have to use Powershell for this. Not because I don’t like Powershell, but mostly because it kind of pierces the vale of the single development experience I’m trying to achieve, I guess. It’s a bit of a context switch to be doing most of the work in WSL, and then for this one thing, need to pop over to Powershell, which also means maintaining duplicate installs of software in Windows land and WSL land. A small thing, for sure, but I’d like to avoid it if possible.

 

Windows IPv6 slow-or-broken: resolved

This is part of a series on moving from desktop Linux back to Windows.

The first post is here. The previous post is here.

Chocolatey slowness?

When I originally posted about package management with Chocolatey, I mentioned two problems I had on a brand new laptop: 1) inability to download large packages; 2) general slowness when downloading.

Turns out, these are not Chocolatey’s problems at all.

Wait… Powershell too?

I noticed when working in Powershell that curl, which just wraps Invoke-WebRequest, was taking a really long time. Simple commands were returning results like this:

PS C:\WINDOWS\system32> Measure-Command { curl https://microsoft.com }

Days : 0
Hours : 0
Minutes : 0
Seconds : 43
Milliseconds : 316
Ticks : 433164486
TotalDays : 0.000501347784722222
TotalHours : 0.0120323468333333
TotalMinutes : 0.72194081
TotalSeconds : 43.3164486
TotalMilliseconds : 43316.4486

It would consistently take about 43 seconds.

Resolution: my IPv6 settings

I narrowed it down to a problem on my network — either the main PC, router, or modem — by trying out the curl command above while connected to my phone’s WiFi hotspot instead of our home router.

I’ll spare the sleuthery for later and cut to the chase:  I resolved the problem two separate ways:

  1. by setting the IPv6 DNS settings in my wireless router (A D-Link DIR-880L); I did not stick with this
  2. ultimately, by finding the broken DNS setting in my primary PC’s IPv6 config. This is the one I went with, but the first might be instructive or useful, so:

In my router’s config, I simply set the Primary and Secondary DNS server settings to Google’s DNS

Once I did that, the IPv6 connection in my router went from Not connected to Connected. My IPv6 readiness score went from 0/10 to 10/10, Powershell curl commands returned to reasonable amounts of time, and Chocolatey was both fast and able to download large packages (300+MB with no problem).

However, I couldn’t let go of something… why would I need to do this anyway? It occurred to me that a very, very long time ago — maybe as far back as 2010, when I first bought our current home PC, I had monkeyed with a DNS setting for some reason or another related to my job at the time.

So I went into the IPv6 settings (here’s how), and sure enough in the DNS tab I had overridden the default with some mumbo-jumbo I’ve long forgotten about. I set it back to the default, undid the changes I had made in the router, and voila, same expected behavior with Powershell, Chocolatey, and IPv6 readiness.

The End. Stop here if you don’t care about how I figured this out.

Sleuthery for those who like stories about troubleshooting

Some people like stories about troubleshooting — I know I love reading them — so here’s the story of troubleshooting this problem.

It is admittedly crazy on first take: that a buried-kinda-deep TCP/IP setting on one computer would not affect that computer, but would affect another computer on the network.

When I noticed the problem with Chocolatey originally, I saw that it was timing out in Invoke-WebRequest. That’s what eventually led me to even be curious about the behavior of curl / Invoke-WebRequest in Powershell itself, outside of Chocolatey. It just took me a few weeks (and a day off of work) to make time to investigate.

I noticed that initial requests were taking about 43 seconds, and a second request would take the same time as well. Then, subsequent requests would complete quickly. After a few seconds of waiting, they’d return to 43 seconds. A very helpful person on the StackOverflow post I made provided a quickie Powershell script to gather some data, which confirmed the above behavior.

I tried curl in Ubuntu via Windows Subsystem for Linux, and it looked fine.

I disabled, and then uninstalled, Antivirus. No effect.

I then connected to my phone’s WiFi hotspot instead of our home wireless, and voila, everything worked fine.

Now that’s interesting. Sounds like a network problem. Let’s try to isolate the various components in the home network and pin one down as the culprit.

I went to our primary PC and tried an Invoke-WebRequest in Powershell, and it was fine, too.

So here’s where I was: 1 computer on the network worked fine, and 1 didn’t. What’s the difference?

Turns out, sometime within the past year, on the home PC, I needed to disable IPv6 for something related to Comcast email and Outlook, and I remembered that.

So on the laptop, I tried disabling IPv6 just as I had on the home PC, and that solved the problem on the laptop.

Fantastic.

Except… why? Why did that work? It made no damn sense. I hated not understanding that.

At first, I tried investigating the problem via the wireless router. I went into the configs and everything was at the default, but I also noticed that it was telling me that IPv6 was Not Connected. Weird. I tried monkeying with a few settings whose purpose I didn’t know, and that made no difference. I then returned it to the default setting and on a whim looked up Google’s DNS IPv6 servers and plugged them in.

After restarting the router, the router showed me Connected for IPv6, and back on the laptop I re-enabled IPv6 and everything seemed to work fine.

To be clear: changing DNS here wasn’t a momentary stroke of brilliance. Often, my troubleshooting strategy boils down to “I wonder what happens if I twist this knob,” and that’s what this was. In my line of work, I’ve seen DNS cause all manner of network goofiness, so this was kind of like “hmmm…. here’s an empty text field related to DNS, and since DNS can act-a-fool, I wonder what happens if I put something legit in it.”

Fantastic-er. Can I be done now?

Why? I mean, sure, it worked, but why would I need to override DNS in the router?  That, also, just didn’t sit right. And I could not let it go.

The only thing I could think of at this point was that there was something wonky about the home PC’s IPv6 settings, and overriding the DNS settings in the router was working around that wonkiness.

I then went into the home PC’s IPv6 settings, and everything looked default.

Clicked “Advanced”, and then “DNS”, and that’s when I saw it: There was a radio button checked and then it had a text box with DNS entries that I have no recollection of setting, and one of the FQDN’s was the internal domain for one of my old jobs. WTF?

So then I returned that setting back to the default, undid the router DNS change, and success!

Though… there’s something else: I’ve had Mac, Linux, Android, iOS, and Windows devices using this network for years and none have hit this problem. Just Powershell. Curious, don’t you think?

Finally, if you’re looking for more of these troubleshooting stories, here’s one from a while back on The Curious Case of the Slow Jenkins Job.

Goin’ back to Windows: Windows Subsystem for Linux

This is part of a series on moving from desktop Linux back to Windows.

The first post is here. The previous post is here.

In short: I would not have moved back to Windows had it not been for the Windows Subsystem for Linux (WSL).

It has made the move from Linux back to Windows so much easier because I can still do the things that I’ve been doing for years, and do every day, quick and easy and familiar: installing and updating development environments (Python, Go, nodejs), databases (PostgreSQL); interacting with source control (git); running apps and tests; running docker clients; etc. So many of the technologies I work with day to date are “Linux/Mac first”, or maybe more accurately “Windows… last”. That was a big motivator for me moving off of Windows in the first place.

WSL is, for me, a game changer when it comes to the local development experience on Windows.

What a world we live in.

What is WSL?

WSL, in short, lets you install Linux environments on Windows without using a virtual machine. Jessie Frazelle has a great write-up on how it works under the hood. For installing and using WSL, the docs are great, too.

Ubuntu on Windows
Ubuntu on Windows

 

What can you do in WSL?

You can run Linux/Mac-first programming environments

Python in WSL

You can install and run services, such as PostgreSQL.

PostgreSQL in WSL

You can call Windows binaries from Linux, and vice versa. Here’s a ridiculous example that you’d never do in real life because the win key or launchy both make running windows binaries so easy, but hey, what the heck:

Running a Windows binary in WSL

 

 

Result of previous command

Note: you need Windows 10 PRO to run Docker for Windows!

You can run a docker client in Linux, talking to Docker containers running via Docker for Windows

Docker client in WSL

You can use a terminal manager like ConEmu — for me, kinda just like iTerm on Mac or Terminal on Ubuntu — and create tabs to your heart’s content (and you can even mix-n-match with linux, cmd, and powershell terminals; and, yes, you can do it all with keyboard shortcuts)

Multiple terminals via ConEmu

You can, of course, tmux

tmux in WSL

You can edit your code in your favorite editor (VS Code is a Windows favorite) and run it on Linux.

VSCode on the left, running Go on the right

Wrapping up

There are, of course, shortcomings and bugs (but how cool is it that WSL itself is on GitHub?!).

I haven’t used it enough in anger yet, and admittedly have not used it heavily enough to have hit any absolute blockers (I’m sure they exist). Though I have brought several of my personal programming projects from Ubuntu onto Windows — all using Go, PostgreSQL, and gulp — and it was all straightforward. Except gulp (natch), b/c of some npm weirdness that took me as long to resolve as it did to bring over the Go apps and databases.

Thus far, though, I am very happy with the local development experience on Windows, thanks in large part to WSL.