Karl Stolley

I was inspired by Scott Hanselman’s blog post on configuring WebFinger for use with Mastodon. The idea is that you leverage a domain in your control to enable people to search for you in the Mastodon fediverse—freeing you from anchoring yourself to a specific Mastodon instance, and also freeing your friends and enemies to find you via an email address or really <any-value-here>@example.com.

I wanted to make it possible to do just that using my new public email address, karl@stolley.dev. If you search your favorite Mastodon instance for that email address, or even ugh-that-guy@stolley.dev, you should see my @stolley@hachyderm.io username pop up in the results.

Creating the webfinger file and putting it in your URL’s /.well-known/ path is only part of the story, however. You need to make sure that your web server or platform knows how to return sensible responses to requests for /.well-known/webfinger.

I’ve got a humble flat-file setup for my WebFinger file. But to get Nginx to serve /.well-known/webfinger correctly, I also added the following location block for the file:

location /.well-known/webfinger {
  types { } default_type 'application/jrd+json';
  add_header 'Access-Control-Allow-Origin' '*';
}

The first thing that block does is ensure that the webfinger file is sent with the correct application/jrd+json MIME type. The funny types { } syntax, with the empty braces, overrides Nginx’s default MIME types in mime.types and sets application/jrd+json as the default MIME type. The call to types also has the effect of setting the content-type HTTP header. For that reason, you should’t attempt to call add_header to set content-type. From experimenting, I found that it and other Nginx header-setting methods resulted in two content-type headers (an HTTP no-no): one content-type header with the Nginx default application/octet-stream and the other with the correct application/jrd+json.

The second thing that location block does is set a permissive CORS header, in keeping with the WebFinger spec (RFC 7033), specifically section 5 on CORS. That little header ensures the widest possible access to the file, including from client-side JavaScript.

To test things out, I made this little call to curl—here showing only the HTTP response headers and contents of my webfinger file:

$ curl -v https://stolley.dev/.well-known/webfinger

# A bunch of stuff snipped here...

< HTTP/2 200
< server: nginx
< date: Wed, 21 Dec 2022 19:01:38 GMT
< content-type: application/jrd+json
< content-length: 659
< last-modified: Wed, 21 Dec 2022 17:58:22 GMT
< etag: "63a3493e-293"
< access-control-allow-origin: *
< accept-ranges: bytes
<
{
    "subject":"acct:stolley@hachyderm.io",
    "aliases":
    [
        "https://hachyderm.io/@stolley",
        "https://hachyderm.io/users/stolley"
    ],
    "links":
    [
        {
            "rel":"http://webfinger.net/rel/profile-page",
            "type":"text/html",
            "href":"https://hachyderm.io/@stolley"
        },
        {
            "rel":"self",
            "type":"application/activity+json",
            "href":"https://hachyderm.io/users/stolley"
        },
        {
            "rel":"http://ostatus.org/schema/1.0/subscribe",
            "template":"https://hachyderm.io/authorize_interaction?uri={uri}"
        }
    ]
}

I’ll get right to it: both the stock Nightwatch configuration file (as of at least Nightwatch v. 2.0.9) and the Nightwatch docs are inaccurate for using the geckodriver web driver (specifically, v. 0.30.0) to run tests. Here is what you need to do, isolated to the firefox environment portion of the test_settings object in a nightwatch.conf.js file:

module.exports = {

  // snip, snip, snippety-snip

  test_settings: {

    // more snipping...

    firefox: {
      capabilities : {
        browserName : 'firefox',
        acceptInsecureCerts: true,
        'moz:firefoxOptions': {
          args: [
            // '-headless',
            // '-verbose'
          ],
          prefs: {
            // 'media.navigator.permission.disable': true,
            // 'media.navigator.streams.fake': true
          }
        }
      },
      webdriver: {
        start_process: true,
        server_path: '',
        host: '127.0.0.1',
        port: 4444,
        cli_args: [
          // very verbose geckodriver logs
          // '-vv'
        ]
      }
    }

  }

};

Curious about what’s going on? I’ll elaborate, property by significant property:

1. The property you want is capabilities, not desiredCapabilities. The capabilities property is what ultimately shipped in the WebDriver specification. While it seems that both geckodriver and chromedriver continue to support the older desiredCapabilities, capabilities is the property you want now and into the future.
2. The acceptInsecureCerts property should be a member directly on capabilities; this too is in the WebDriver spec. And in testing today, it seems that some combo of Nightwatch and geckodriver ignore acceptInsecureCerts in any other location.
3. For Firefox-specific options (like command-line args and browser about:config prefs), the property to use is moz:firefoxOptions. That is shipping in the latest default nightwatch.conf.js file, as is the parallel goog:chromeOptions for chromedriver. But if you’ve got a legacy configuration file—like I did—you’ll want to update the old chromeOptions property, too.
4. For local testing, you’ll want to set an IPv4 address to your local loopback for the host property on webdriver. On the Macs I was working on (one Big Sur, one Monterey), localhost was resolving to the IPv6 ::1 adddress. Geckodriver was refusing the connection on that, as I discovered by running wget out of desperation. wget then retried on 127.0.0.1:

     $ geckodriver
     1648330755779  geckodriver  INFO  Listening on 127.0.0.1:4444
     # separate terminal process, with geckodriver still running:
     $ wget http://localhost:4444/
     --2022-03-26 16:49:18--  http://localhost:4444/
     Resolving localhost (localhost)... ::1, 127.0.0.1
     Connecting to localhost (localhost)|::1|:4444... failed: Connection refused.
     Connecting to localhost (localhost)|127.0.0.1|:4444... connected.
     HTTP request sent, awaiting response... 405 Method Not Allowed
     2022-03-26 16:49:18 ERROR 405: Method Not Allowed.
   

   The 405 error is expected: there’s no meaningful resource on the geckodriver for GET /. But the Nightwatch test-runner (I assume) did not attempt such a retry. Instead, it left behind a seemingly impossible error message of Failed to connect to GeckoDriver on 127.0.0.1 with port 4444.

5. Bonus information: geckodriver now supports parallel testing, where two browsers can be open once! This is important for me, as I’m using Nightwatch to test out the code I’ve been writing for my book on WebRTC. I’ll save the full instructions for how to pull off parallel tests in full for another post, but essentially, you just need two different environments (I have firefoxActive and firefoxPassive, rather than just firefox). And each environment must have its own webdriver process on its own port. So 4444 (the default) on one, and say 4445 on another. When you go to run nightwatch, pass in the -e argument with the two environments: npx nightwatch -e firefoxActive,firefoxPassive.

With #PragProWriMo off and running this November, I wanted to share some of my thinking about becoming a better writer. Not by looking at the best of how I write, but at the worst. Motivation can only get you so far if you’ve got a hang-up about your writing.

Simple truths: Everyone writes. Some of us enjoy it. Most of us don’t. And consciously or not, almost everyone has some kind of hang-up about writing. It might represent a mild difficulty, or be something as serious as a root cause of writer’s block. At its worst, it might even be a nagging voice that tells you Sorry, you’re not a writer.

Let me tell you about my writing hang-up. I’ll even demonstrate it for you:

I’m a writer who’s still recovering from graduate school, where you’re positively reinforced—and often richly rewarded with praise and publications—to write extraordinarily complex sentences (no matter what the topic); the more complex the sentence, the more the need to avoid, as long as possible, any kind of terminal punctuation: a sentence-ending period, for example, completes a thought, and no thought is ever complete—so eventually you end up using every piece of punctuation on your keyboard (parentheses, commas, em-dashes, colons, semicolons), and the sentence you’ve constructed spans several lines, and it ends up in a completely different place from where it started.

That’s my hang-up, on full display in that sentence, right there. If you read that whole thing, I’m so sorry.

Sentences like that are what I’ve been working on hardest to improve about my writing. I’ve been making some progress, I guess, because it was actually kind of painful instead of natural to write that terrible, embarrassing demonstration above.

The worst thing about my hang-up with avoiding terminal punctuation is that I write sentences that are beyond saving. They’ve got so many twists and turns that there’s very little that can be saved or properly edited. Those kinds of sentences take on a life and trajectory all their own. I can’t control them any better when I’m trying to fix them than I could on a first draft.

You’ve probably heard writerly advice like “Just let it out! Draft and just keep going!” I’ve heard that, too, but it has never worked for me. It just produces the kinds of awful sentences like I demonstrated above. “Just keep going” is the worst thing I can let myself do, because my hang-up is that I will just keep going. And going. And going. Real Energizer Bunny stuff.

So I edit myself as I write. (Note that self-editing is no substitute for having someone else, especially a professional, edit your writing.) And by constantly working on my writing, I have become very familiar with the many kinds of bad habits I have. After I’ve fixed the worst ones, I can see others: I make myself keep my subjects near my verbs. I try to be sparing about dropping fancy inductive sentences, and instead go with a straight-up subject-verb-object pattern. Usually.

At best, the end result is improved clarity. Or at least writing that isn’t getting in the way of the material I’m trying to convey. There’s plenty of confusion in the mix with technical subject matter already—no matter how skilled a writer you are.

Maybe you’ve got anxiety about your command of grammar (everyone does). Maybe you’re laboring under a bunch of silly myths and lies about writing that you were taught in school: “Never use I,” “Never start a sentence with a conjunction,” or “Never end a sentence with a preposition.” Those are all lies. Maybe you think your writing sounds boring, or maybe you think you’ve not got a big enough vocabulary.

But here’s the thing about a writing hang-up: you’ve got to acknowledge it and hit it head on, whatever it is. Don’t let it become the excuse not to write. Let it be the thing you decide to fix—not before you write, but as you write.

Take my hang-up with long and winding sentences. I’ve found a technique for countering it. And that’s to read more writing by authors who do exactly the opposite of whatever I’m trying to fix in my own writing. Less David Foster Wallace, more Virginia Woolf.

But reading isn’t a good excuse not to be writing, either. To get myself primed to write, I often read just a few punchy, staccato sentences by someone like Hunter S. Thompson, something like this short passage from his classic “The Kentucky Derby is Decadent and Depraved”:

Steadman is now worried about fire. Somebody told him about the clubhouse catching on fire two years ago. Could it happen again? Horrible. Trapped in the press box. Holocaust. A hundred thousand people fighting to get out. Drunks screaming in the flames and the mud, crazed horses running wild. Blind in the smoke. Grandstand collapsing into the flames with us on the roof. Poor Ralph is about to crack. Drinking heavily, into the Haig.

That’s a whole paragraph with only two pieces of non-terminal punctuation (a couple of commas). As an added bonus, it’s funny. It’s visual. It creates a scene. And that activates the better, more creative parts of my brain.

Writing to teach others about technical subjects requires stringing together a lot of complex ideas. A short passage like Thompson’s is enough to remind me that I can have a complete thought or idea (like a burning clubhouse at the Kentucky Derby) and that I can convey it in a number of short, punchy sentences. Not everything can be short and punchy, but changing up from the monotony of longer sentences makes a big difference.

I work on my writing hang-up as a pet project of personal and professional improvement. But that’s really secondary to what I’m trying to do for readers. Every period is a place to rest. A short sentence invites a reassuring, I’ve-got-this re-read for uncertain readers. If readers can retread small or at least manageable steps, they’ll be able to follow me wherever I try to take them. There is no need to try to lead someone through the maze of one massive sentence.

Or so I keep having to tell myself.

It’s the end of May, and I’m now about ten weeks into the contracted part of this project for Pragmatic Programmers. And in the last week, the book has grown from its original proposed 6 chapters at 150 pages to 8 chapters that will weigh in at around 250 pages. That gives me a lot more room to work and should prevent anything vital from having to be cut.

As I put the finishing touches this week on the work for my first major milestone—the three-chapter publisher’s review, which I’d originally planned for mid-June—the table of contents its looking like this, minus an introductory setup chapter that’ll be part of the frontmatter and an appendix showing a signaling channel written in pure WebSockets (the book itself uses Socket.IO in its recently released version 4):

1. Working with a Signaling Channel. Readers set up an interface in HTML, CSS, and JavaScript to manage a basic video-chat app and wire up a pre-built signaling channel and its callbacks. A high-level description of WebRTC contextualizes the signaling channel’s purpose and limited features.
2. Establishing a Peer-to-Peer Connection. Picking up right where the last chapter left off, readers learn to request permissions for camera/mic access and display the resulting stream. They then go on to create a peer connection using the “perfect negotiation” pattern described in the WebRTC specification and build out the rest of the basic video-chat app.
3. Handling Data Channels. Media streaming is WebRTC’s most famous feature, but it’s data channels that really open up the possibilities for all kinds of real-time, peer-to-peer apps. Readers add a video-filter function to the app built over the first two chapters and then build a complete text-chat feature.
4. Streaming Complex Data. Data channels can handle more than simple strings. This chapter opens with building fallbacks into the perfect-negotiation code so they can more confidently test their work, including especially with data channels, on browsers that have poorer implementations of WebRTC (ahem, Safari). Readers work with sending and receiving JSON strings to build out the chat feature further, and then turn to streaming binary data—as either Blobs or ArrayBuffers, using some clever feature-detection—to share images in the chat. Readers build a standalone peer-to-peer file-sharing app to close out the chapter.
5. Building Peer-to-Peer Interfaces. The first four chapters cover all the fundamentals of WebRTC. This chapter looks more closely at real-time, peer-to-peer interface design—all framed by accessibility. Readers learn to build interfaces that take advantage of semantic HTML and ARIA attributes, while also giving users greater control over real-time interfaces, such as arranging interface elements in relation to their cameras.
6. Managing Multi-Peer Connections. This chapter moves from one-to-one to many-to-many WebRTC apps. Sticking to the browser-based focus of the book, it introduces a mesh-network architecture for connecting three or more peers in a single call. Readers also learn the practical and theoretical upper limits to the number of simultaneous peer connections, and begin to work with the RTCPeerConnection.getStats() interface.
7. Optimizing Media Streaming. Readers take a deep dive on different media APIs in this chapter, and learn to optimize streaming media based on local and remote statistics and even the simple appearance of media streams in the browser (how big does that video stream need to be? how small can it get?). The chapter also looks at current and forthcoming methods for optimizing images sent over data channels, including the exciting but currently not universally supported self.createImageBitmap() method.
8. Deploying WebRTC Apps to Production. This final chapter of the book looks at what it takes to deploy a WebRTC app into production. Readers learn about using public STUN servers as well as setting up a private STUN/TURN server, with the TURN component as a fallback for when a direct peer-to-peer connection cannot be established.

There are a lot of topics packed into chapters six, seven, and eight especially. So I wouldn’t be surprised to discover a topic that deserves its own chapter—either on my own as I write or from feedback that’ll come in from the technical and beta reviews later in the summer.

The first three chapters are draft complete, apart from some intro and outro work that I need to do today. And I’ve got good chunks of the fourth and sixth chapters coming along, in addition to a growing number of topics for the fifth chapter, on interfaces and accessibility.

Put another way, this is starting to feel more like a book to me—and not just a collection of stuff I’ve written. Considering that the spring semester just ended for me a little over two weeks ago, I’m pretty happy with where I am. But I’m hoping to pick up the pace as I recover from this weird academic year and settle into the summer.

For the last few weeks, the brisk progress I’d been making on my WebRTC book basically ground to a halt thanks to my “day job” (as my development editor whimsically puts it). My day job is being a professor, and there’s really no worse time of year than mid-April until final grades get turned in. That period isn’t just about helping students over the finish line, but also accommodating a bunch of panicky colleagues and administrators scheduling last-minute meetings before people start disappearing for the summer.

Anyway. While I’ve not written much at all on the manuscript, I decided to take what time I could manage to work on my own organization, particularly all of the thousands of lines of source code I’m writing that’ll accompany the book.

Writing code for yourself or for your job is usually a bit different from writing code to teach others. Even the code that I write in order to teach in the classroom is different from code I write to accompany books and articles: my usual way of prepping for code-intensive classes is to do a dry run and then have skeletal notes or sometimes more complete examples to refer from, if I’m teaching something extra complex and am likely to forget some small but essential detail in the middle of class. I don’t like just walking through canned, finished examples in class: live coding is how I roll.

In an instructional setting, even if its online because of the pandemic, there’s always the chance to go back and fix something later or, in the best cases, improve something on the spot in response to a student question or point of confusion. Writing code live makes for a better class than a shitty slide deck with completed examples, I think, but it also helps students sort of wrap their heads around the timescales and especially the process of writing code. As every developer knows, the process is never linear.

But when you’re writing code for a book or an article, you don’t have the luxury of refining things once the piece is published. Throughout the writing process, you’re building a foundation and guiding someone down a path that you, as the author, will have finished before the reader even starts out. And that means that you’ve got to do a lot more planning, as an author and a developer, to figure out what the reader’s journey is going to look like.

To smooth the way for themselves, the authors of some books and articles—too many books and articles, I’ll add—basically set up the code equivalent of a straw person: early examples so amateurishly written and poorly formed that they’re basically laughable. Of course, just like the straw-person argument, it’s easy to swoop in as the hero-author and make yourself look amazing by fixing crappy, amateurish code.

I don’t like that approach, though, and I avoid it in my own work. And this is why: from years of both reading a lot of these books as well as teaching them, I can say that the first examples readers see tend to make the deepest impressions. I notice this in my own work, when I make a mistake that a book deliberately showed me and only then admonished me not to make. I see the same thing happen in student work too, either when they repeat a mistake that they were shown in a book or a mistake that I showed off in class before clarifying it as a mistake. That pattern is annoying for readers, too: Why did we just spend all that time talking about this thing that you should never do?

So rather than starting from crappy code, I like to start with reasonably good code—which, just like the running copy of anything I write, is never the code that I draft.

The trick I’ve found, and the thing I’ve been working on for the last few weeks with my book, is to write the absolute best code you can on your own. And then you dial it back a few notches from there. Maybe that fancy ternary operator has to go, or that little shortcut method or piece of syntactic sugar: anything that would take time and space too much to explain to readers, especially when they’re first starting out. It doesn’t mean that you won’t ultimately get the code whipped into that shape, but that you don’t have to start there—and leave readers feeling lost by all the ancillary, look-how-smart-the-author-is stuff that can wait.

I do think it’s useful, however, to point out common mistakes after walking through a piece of code—especially readers are expected to write it out themselves and run it. Beginners trying to pass a callback function in by reference in JavaScript, for a simple example, will very often include parentheses, so that the correct code on('event', callbackFunction); gets miswritten as on('event', callbackFunction());. It’s an easy mistake for beginners to make, because they’re used to seeing either the function definition— function someFunction() { } —or the place in the code where the function is called: someFunction().

The catch, of course, is when you as an author make a mistake in your own code. Alongside all of this, I’ve also been upping my automated testing game. (That’s included me bringing Nightwatch.js to the party. Testing WebRTC apps means having multiple, simultaneous browsers open for the duration of the test. Chrome can do that, but Gecko’s web driver implementation seems to be lagging somewhat. But this is all material for a whole other post.) Once I’ve got good-enough draft examples, I can’t help but continue to improve them. Sometimes the improvement is just about the code itself, but other times it’s about making the text read more gracefully. But even for books or articles, there’s no confidence in refactoring code without good test coverage. You’ve really not experienced BDD in full until you’re writing tests on code that you know will evolve a particular way over the length of a book.

One of the things I’m finding challenging about writing WebRTC code over a book-length work is that ideally the signaling and connection logic works completely separately and independently from any application logic. The application logic is what carries the examples throughout my book. At the same time, it’s too much to ask readers to write, say, fallbacks for older browsers in the signaling and connection code when they’re just getting started. So it’s necessary for me to keep track of gradual improvements to be made to the signaling and connection logic, even as I’m working through different examples with readers as they progress deeper into the book.

The latest strategy I’ve employed is to write the best, most thoroughly tested examples I can before I even begin to write a chapter. Then, as I write the chapter, I’m re-writing the completed example just as I expect a reader will. That frequently gives me insight as to the fancier or more complicated parts of my completed examples that I can save to guide readers through later.

So, that’s where I am. I’m ready to get back to churning out the running content of the book. And while I wish I had a few thousand more words to show for the last few weeks, I’m feeling pretty good that this code work is going to pave the way for me to better focus on how the book is put together—and how readers will work through it.