Hands-On LSP Tutorial: Building a Custom Auto-Complete

What we'll build in this tutorial

Today, we'll build out completion for linking to GitHub issues and PRs.

Developers can use this completion to reference issues and PRs from their code comments and commit messages.

What is a Language Server?

Language servers let you meet developers where they are: their editor. They give you the power of in-editor tooling without rebuilding your tools for each editor you want to support. They can implement features like completion, go-to-definition, code actions, and more.

The Language Server Protocol (LSP) specifies how a language server should interface with an editor.

Approach

1. We'll build a language server prototype with hard-coded completion choices.
2. We'll wire this to our editor and test it out.
3. We'll replace the hard-coded choices with dynamic content from GitHub's API.

Prototyping our Language Server

We'll build this demo in Typescript.

We install a few dependencies to handle the language server implementation details. We'll also install node-fetch to help us make API requests.

npm install vscode-languageserver vscode-languageserver-textdocument vscode-languageclient node-fetch@2
npm install --save-dev @types/vscode @types/node-fetch

(Don't let the vscode bits confuse you. While Microsoft publishes these packages, your language server will be editor-agnostic.)

Now, we can start writing our code. We'll start with distilled boilerplate code borrowed from the Visual Studio Code Language Server Extension Guide.

import {
  createConnection,
  InitializeResult,
  TextDocumentSyncKind,
  TextDocuments,
} from "vscode-languageserver/node";
import { TextDocument } from "vscode-languageserver-textdocument";

const connection = createConnection();

const documents: TextDocuments<TextDocument> = new TextDocuments(TextDocument);

connection.onInitialize(() => {
  const result: InitializeResult = {
    capabilities: {
      textDocumentSync: TextDocumentSyncKind.Incremental,
      // Tell the client that the server supports code completion
      completionProvider: {
        // We want to listen for the "#" character to trigger our completions
        triggerCharacters: ["#"],
      },
    },
  };

  return result;
});

// Make the text document manager listen on the connection
// for open, change and close text document events
documents.listen(connection);

// Listen on the connection
connection.listen();

After our imports, we set up a connection and some documents. The connection handles the communication between the editor (the client) and our code (the server). documents is a store of open files in our editor. documents stays in sync when files open, close, and change.

When the editor starts our server, onInitialize is fired. The editor sends some params describing what it can do (and what it wants our server to do), but we're not using those here, so we'll omit them. We reply with a list of things we can do (primarily, we offer completions triggered by #).

We tell documents to listen on the connection and the connection to listen for communication from the editor.

Our server could now run hosted by the editor (more on that later), but it doesn't do anything useful yet.

Let's add the initial hard-coded completion after our onInitialize.

connection.onCompletion(() => {
  return [
    {
      label: "Cannot set properties of undefined (setting 'minItems')",
      data: "https://github.com/facebook/docusaurus/issues/9271",
    },
    {
      label: "docs: fix typo in docs-introduction",
      data: "https://github.com/facebook/docusaurus/pull/9267",
    },
    {
      label: "Upgrade notification command does not copy properly",
      data: "https://github.com/facebook/docusaurus/issues/9239",
    },
  ];
});

Now, we're ready to test this out.

Wiring this up to our Editor/IDE

While the LSP standardizes how to implement Language Servers and how they talk to editors, there's no good standard for making your editor aware of a language server so that it is ready to be used. For this post, I'll cover VS Code and Neovim.

How to connect your Language Server to VS Code

You connect a language server to VS Code by creating a thin extension layer. To make things as easy as possible for you, I've published a Minimum Viable VS Code Language Server Extension. Follow the directions in that repo and replace the entire contents of server/src/server.ts with the code we're writing in this post.

After following those directions, we'll also need to install node-fetch for the server since the node version packaged with VS Code is too old to include native fetch support.

cd server && npm install node-fetch@2 && npm install --save-dev @types/node-fetch && cd -

Now, you should be able to test your extension with the steps mentioned here. You should be able to open a new file, type #, and then see something like this:

How to connect your Language Server to Neovim

For Neovim, you pass some configuration to vim.lsp.start. For testing, I've put the following in ~/.config/nvim/after/ftplugin/ruby.lua. Note that this location means it only runs in ruby files. Feel free to start the server elsewhere.

local capabilities = vim.lsp.protocol.make_client_capabilities()

vim.lsp.start {
    name = "test-ls",
    cmd = {
        "npx", "ts-node",
        "/Users/ship/src/lsp-example-completion/server.ts",
        "--stdio"
    },
    capabilities = capabilities
}

I'm using npx ts-node to run my server.ts without an intermediate build step, but you can use whatever approach you prefer.

You should be able to open a new file, type #, and then see something like this:

If you encounter any trouble, check the output of :LspLog

Dynamic Completions

GitHub provides a GraphQL API. Get a GitHub access token and have it handy. It appears a classic token with "repo" permissions is required to get the search query to work correctly (at the time of writing).

Here's a query to search the Docusaurus repo for issues and PRs that include the word "bug.”

{
  search(query: "repo:facebook/docusaurus bug", type: ISSUE, first: 10) {
    edges {
      node {
        ... on PullRequest {
          title
          number
          body
          url
        }
        ... on Issue {
          title
          number
          body
          url
        }
      }
    }
  }
}

We could replace repo:facebook/docusaurus with owner:facebook to scope search every repo owned by Facebook (that we have access to).

You can replace the searchScope with your organization or repo name when applying the changes below. Be sure to swap out the placeholder YOUR_GITHUB_ACCESS_TOKEN with your token. In a future post, we'll cover language server configuration to pass in our search scope and access token.

import fetch from "node-fetch";

type Edge = {
  node: {
    title: string;
    number: number;
    url: string;
    body: string;
  };
};

type Response = {
  data: { search: { edges: Edge[] } };
};

// Replace with your real GitHub access token
const accessToken = "YOUR_GITHUB_ACCESS_TOKEN";

const searchScope = "repo:facebook/docusaurus";

const search = async (input: string): Promise<Response> => {
  const query = `
  {
    search(query: "${searchScope} ${input}", type: ISSUE, first: 10) {
      edges {
        node {
          ... on PullRequest {
            title
            number
            body
            url
          }
          ... on Issue {
            title
            number
            body
            url
          }
        }
      }
    }
  }
`;

  const response = await fetch("https://api.github.com/graphql", {
    method: "POST",
    body: JSON.stringify({ query }),
    headers: {
      Authorization: `Bearer ${accessToken}`,
    },
  });

  if (!response.ok) {
    const error = {
      status: response.status,
      statusText: response.statusText,
      body: await response.json(),
    };

    connection.console.error(
      `Error fetching from GitHub: ${JSON.stringify(error)}`,
    );
  }

  return await response.json();
};

// ...

connection.onCompletion(async () => {
  const input = "bug";

  const json = await search(input);

  const items = json.data.search.edges.map((edge) => {
    return {
      label: `#${edge.node.number}: ${edge.node.title}`,
      detail: `${edge.node.title}\n\n${edge.node.body}`,
      insertText: edge.node.url,
    };
  });

  return {
    isIncomplete: true,
    items,
  };
});

Note the isIncomplete: true here. This tells the editor that the list we're returning isn't exhaustive and might change over time.

Our language server works!

But it uses our hard-coded search string "bug" instead of completing based on our input. Let's fix that.

connection.onCompletion takes a CompletionParams argument that looks something like this:

{
  "textDocument":{"uri":"file:///private/tmp/example_file.rb"},
  "context":{"triggerKind":3},
  "position":{"line":2,"character":11}
}

With this information and the documents we initialized earlier, we have everything we need to understand what the user was typing.

connection.onCompletion(async (params) => {
  const doc = documents.get(params.textDocument.uri);

  if (!doc) {
    return null;
  }

  const line = doc.getText().split("\n")[params.position.line];

  // return early if we don't see our trigger character
  if (!line.includes("#")) {
    return null;
  }

  const input = line.slice(line.lastIndexOf("#") + 1);

// ...

After selecting our completion choice, you’ll notice that the # character hangs around. How do we prevent this? And why do we need a trigger character anyway?

If we didn't have a trigger character, this completion would trigger constantly as we type any code. That's wasteful for API requests and means we're doing unnecessary work -- we only want to show issues and PRs in very specific scenarios. So, the behavior should be opt-in, and a trigger character is beneficial.

To prevent the # from hanging around, we can replace our naive insertText with a TextEdit. TextEdits allow you to specify a range in the document to replace with new text. We'll modify our completion item formatting function:

const items = json.data.search.edges.map((edge) => {
  return {
    label: `#${edge.node.number}: ${edge.node.title}`,
    detail: `${edge.node.title}\n\n${edge.node.body}`,
    textEdit: {
      range: {
        start: {
          line: params.position.line,
          character: line.lastIndexOf("#"),
        },
        end: {
          line: params.position.line,
          character: params.position.character,
        },
      },
      newText: `${edge.node.url}`,
    },
  };
});
// ...

With this change, we overwrite the # so that only the new content remains.

Final code

import fetch from "node-fetch";
import {
  createConnection,
  InitializeResult,
  TextDocumentSyncKind,
  TextDocuments,
} from "vscode-languageserver/node";
import { TextDocument } from "vscode-languageserver-textdocument";

const connection = createConnection();

const documents: TextDocuments<TextDocument> = new TextDocuments(TextDocument);

type Edge = {
  node: {
    title: string;
    number: number;
    url: string;
    body: string;
  };
};

type Response = {
  data: { search: { edges: Edge[] } };
};

// Replace with your real GitHub access token
const accessToken = "YOUR_GITHUB_ACCESS_TOKEN";

const searchScope = "repo:facebook/docusaurus";

const search = async (input: string): Promise<Response> => {
  const query = `
  {
    search(query: "${searchScope} ${input}", type: ISSUE, first: 10) {
      edges {
        node {
          ... on PullRequest {
            title
            number
            body
            url
          }
          ... on Issue {
            title
            number
            body
            url
          }
        }
      }
    }
  }
`;

  const response = await fetch("https://api.github.com/graphql", {
    method: "POST",
    body: JSON.stringify({ query }),
    headers: {
      Authorization: `Bearer ${accessToken}`,
    },
  });

  if (!response.ok) {
    const error = {
      status: response.status,
      statusText: response.statusText,
      body: await response.json(),
    };

    connection.console.error(
      `Error fetching from GitHub: ${JSON.stringify(error)}`,
    );
  }

  return await response.json();
};

connection.onInitialize(() => {
  const result: InitializeResult = {
    capabilities: {
      textDocumentSync: TextDocumentSyncKind.Incremental,
      // Tell the client that the server supports code completion
      completionProvider: {
        // We want to listen for the "#" character to trigger our completions
        triggerCharacters: ["#"],
      },
    },
  };

  return result;
});

connection.onCompletion(async (params) => {
  const doc = documents.get(params.textDocument.uri);

  if (!doc) {
    return null;
  }

  const line = doc.getText().split("\n")[params.position.line];

  // return early if we don't see our trigger character
  if (!line.includes("#")) {
    return null;
  }

  const input = line.slice(line.lastIndexOf("#") + 1);

  const json = await search(input);

  const items = json.data.search.edges.map((edge) => {
    return {
      label: `#${edge.node.number}: ${edge.node.title}`,
      detail: `${edge.node.title}\n\n${edge.node.body}`,
      textEdit: {
        range: {
          start: {
            line: params.position.line,
            character: line.lastIndexOf("#"),
          },
          end: {
            line: params.position.line,
            character: params.position.character,
          },
        },
        newText: `${edge.node.url}`,
      },
    };
  });

  return {
    isIncomplete: true,
    items,
  };
});

// Make the text document manager listen on the connection
// for open, change and close text document events
documents.listen(connection);

// Listen on the connection
connection.listen();

We've only scratched the surface of what language servers can do. There's still plenty of the LSP left to explore.

Read the next article Hands-On LSP Tutorial: Configuration.

If you'd like to be notified when I publish more LSP content, sign up for my newsletter.