TypeScript and npm modules

The new big thing at work is TypeScript¹. I like the idea of bringing more potential for compile time checking and better tooling to JavaScript, and with all these new fangled features like type inference it should hopefully not bring too much of overhead to the code.

Recently I’ve also been looking at doing some better integrationy stuff at home. We got a new flatmate who brought along a Chromecast and have since switched to using Plex as our main media server. The Plex web UI and mobile apps both allow streaming videos/music directly from Plex to the Chromecast. One integration area this is currently lacking compared to the old setup was being able to see which tv/movies are available and start them playing in XBMC directly from the trakt web site².

After a quick look at the Plex API I decided that this shouldn’t be too hard to implement myself³, and while I’m at it I may as well learn about writing code in TypeScript. Learning the infrastructural areas around TypeScript will be especially important as getting them right the first time once my team actually starts using TypeScript at work will save a lot of hassle in the future. Looking round at tools I’d heard mentioned by co-workers I thought it should be easy enough to write a Plex API client library in TypeScript, using npm modules like rest as a base, pulling in .d.ts⁴ files from the DefinitelyTyped repository using tsd and publishing the client as an npm module including compiled .js files along with the .d.ts files so it can be consumed by other TypeScript libraries easily. With this client in place creating the small amount of UI for the trakt integration would be a cinch.

Once I actually started to build this base api client I quickly started running into issues that made it not very nice to work with. The very first one was the lack of and quality of .d.ts files available on DefinitelyTyped, this is sort of expected since TypeScript is a relatively new language and is easy enough to fix by writing these .d.ts files and contributing them back. The second issue was how to actually bring these .d.ts files into the client. The recommended method is to add a reference line to your code to pull in these definitions e.g. if you have some source file lib/client.ts and have installed your .d.ts files into the default typings folder with a top-level tsd.d.ts to reference them you would need to add

/// <reference path="../typings/tsd.d.ts" />

to lib/client.ts. Straight away this seems wrong to me, why are you having to manually include type definitions when doing something as simple as loading a module you use? Why is this not taking care of automatically by the compiler?

It gets even worse once you look at using a node module written in TypeScript by another library written in TypeScript. For this example lets say I’d finished the Plex API client with the following source file structure⁵:

plex-api
├── index.ts
├── lib
│   └── client.ts
├── package.json
├── tsd.json
└── typings
    ├── rest
    │   └── rest.d.ts
    └── tsd.d.ts

Prior to publishing this would be compiled to .js files by tsc, this will allow consumption by normal JavaScript libraries. At the same time the .d.ts files can be generated and added to the module to provide the necessary type annotations for any TypeScript libraries consuming this module. This would result in the following module file structure:

plex-api
├── index.d.ts
├── index.js
├── lib
│   ├── client.d.ts
│   └── client.js
├── package.json
└── typings
    ├── rest
    │   └── rest.d.ts
    └── tsd.d.ts

Now, any normal JavaScript consumers of this library can simply

var plexApi = require('plex-api');

as usual. For any TypeScript consumers however it seems like they’d need to

/// <reference path="../node_modules/plex-api/index.d.ts" />
import plexApi = require('plex-api');

Except that won’t actually work. The .d.ts files generated by tsc are source files that define an external module (§11.1). These sorts of module definitions only work when the compiler’s pull type resolution resolves an external module reference to the file directly⁶. This doesn’t occur when they’re pulled in by a reference directive, although I can find very little information on what exactly should happen when a reference directive is encountered, §11.1.1 contains the only reference to it I can find and simply says

A comment of the form /// <reference path="…"/> adds a dependency on the source file specified in the path argument. The path is resolved relative to the directory of the containing source file.

Anyway, the only sorts of .d.ts files that seem to work well in reference directives are ones that contain Ambient External Module Declarations (§12.1.6). These declarations do not cause the source file to be considered an external module and instead allow it to be a part of the global module (§11.1). When the compiler later attempts to resolve a top-level external module name (like when resolving import plexApi = require('plex-api');) then if there is an ambient external module declaration that matches it will be preferentially returned before attempting to find a file defining the module (§11.2.1).

This works well for definitions pulled in by DefinitelyTyped, since they’re being handwritten for the modules it’s easy enough to write them as ambient external module declarations⁷. For modules generated from .ts files though this is a problem. It’s a non-trivial conversion to take all the output .d.ts files and generate them correctly as ambient external module declarations.

There is still another issue however. Even if you convert the definition files into ambient external module definitions you start running into dependency issues. Lets say you are writing another module that depends on the fixed up plex-api module (which also happens to minify all its JavaScript into index.js). This module also has to access another ReST service, but it’s such a small access that they decided not to create a separate client library for this service and instead write it directly into the module. So as well as depending on the plex-api module they depend directly on the rest module. The file layout for this situation would be something like:

my-cool-module
├── node_modules
│   ├── plex-api
│   │   ├── index.d.ts
│   │   ├── index.js
│   │   ├── node_modules
│   │   │   └── rest
│   │   │       ├── index.js
│   │   │       └── package.json
│   │   ├── package.json
│   │   └── typings
│   │       ├── rest
│   │       │   └── rest.d.ts
│   │       └── tsd.d.ts
│   └── rest
│       ├── index.js
│       └── package.json
├── index.ts
└── typings
    ├── rest
    │   └── rest.d.ts
    └── tsd.d.ts

With file contents:

my-cool-module/node_modules/plex-api/typings/rest/rest.d.ts:
my-cool-module/typings/rest/rest.d.ts:

declare module "rest" {
  // Module declarations
}

my-cool-module/node_modules/plex-api/index.d.ts:

/// <reference path="./typings/tsd.d.ts" />
declare module "plex-api" {
  // Module declarations
}

my-cool-module/index.ts:

/// <reference path="./typings/tsd.d.ts" />
/// <reference path="./node_modules/plex-api/index.d.ts" />
// All the cool codes.

The issue with this is that the two modules will be referencing different files that both declare the ambient external module “rest”. As soon as the second one is loaded the compiler will emit a duplicate declaration warning. If instead the plex-api module were to not distribute its dependency typings you’d get the opposite issue, any library that uses plex-api would have to include all the dependencies in its own typings folder.

My next post should have a proposed solution to this issue. For a preview take a look at this test github repo and this TypeScript pull request.

---

1. Well, at least one of the big new things, and it’s not really that big or that new, I guess it’s really just a thing. ↩

2. Provided by the XBMC Trakt.TV Remote Chrome extension. ↩

3. Other than a complete lack of documentation for the Plex API. ↩

4. TypeScript Definition files, these allow using node modules written in pure JavaScript while still retaining type information. ↩

5. Pretend there’s a whole tree of files under lib, just putting a single file there makes this example easier. Also likely to be many more typings than just the rest one. ↩

6. Don’t ask me to explain what this means, it’s just based on the source file and method names that needed changing to workaround this limitation. I might have more details by the time I post about the solution. ↩

7. Disappointingly there’s a lot of definitions available that don’t follow this standard however… ↩