Creating Node.js Microservices with Ease Using Stdlib

Microservices and serverless architectures are all the rage in the software industry. After working with Polybit’s amazing stdlib platform, I am clearly seeing the value of this promising technology! Today, I will introduce you to stdlib. I encourage you to work alongside me as we leverage stdlib to build a microservice that we can consume in a variety of contexts. Let’s get started with this fabulous technology!

What is stdlib?#

Polybit’s stdlib is a function-as-a-service (FaaS) platform for creating modular, scalable microservices using Node.js. It provides a great deal of capability, and it is lots of fun too. 🙂

Stdlib is considered a “serverless architecture”. The first few times I heard the term “serverless” bandied about, I scoffed at this concept. I consider myself to be fairly astute, and I know there is no such thing as cloud architectures that run without servers. Or…have I missed something along the way? To break the news to everyone (brace yourselves for this one…), serverless architectures actually use servers. We call these architectures “serverless” since they run on a system where we don’t own the infrastructure and we can therefore simply focus on getting the job done writing code. We don’t have to stress about operating system patches, or even fret about keeping the application infrastructure up to date. This is indeed a huge advantage and productivity enhancer!

As described in the stdlib documentation:

stdlib consists of three components:

• A central registry for microservices
• A distribution platform for hosting at scale
• A development framework for package management and service creation

It is the fastest, easiest way to begin building microservices on your own or with a team, and currently supports Node.js 6.9.1. The distribution and hosting platform for execution of your services is built on AWS Lambda, ensuring both the scale and reliability you would expect for production-ready services.

By the way, the main stdlib documentation page is well written and provides a great introduction to stdlib as well. You should consider reading this documentation before or after reading this guide to reinforce concepts. My guide here seeks to expand on the main stdlib documentation and walk through the creation of a real-world microservice beyond the requisite “hello world” application that we all know and love. 🙂

Paving the way for stdlib#

Ok, enough background info. Let’s get rolling and create an awesome microservice in the cloud that we can use and share with our family, friends, and coworkers. Here we go!

As a first step, install Node.js from here, if you do not already have it installed. You will need to be running Node v6.x or higher to get in the game.  If you wish to run on a Raspberry Pi, follow the steps in my Beginner’s Guide to Installing Node.js on a Raspberry Pi.

Next, from the terminal, enter the following to install the nice, concisely named lib.cli package globally on your system:

npm install -g lib.cli

Very good – we’re ready to build our first service.

Our microservice – find the city using GPS coordinates#

Let’s move a few steps beyond “hello world” and build a microservice in stdlib to tell us the name of the city when GPS coordinates are supplied. My apologies in advance to all of you, my friends, in other countries, since we will only be able to find cities in the United States. Perhaps you can employ these concepts, and find cities in other parts of the world as well.

First, get out your pen and paper (real or virtual), and go to this GPS Coordinates website, select a place in the USA to use for testing your (yet to be written) microservice, and jot down the GPS coordinates. San Diego is near and dear to my heart, so I will be using the following GPS coordinates for my microservice:

• latitude: 32.72
• longitude: -117.16

Please observe that I have rounded off the GPS coordinates to two decimal digits. This precision will more than suffice for locating a city given the GPS coordinates.

Ok, take a moment to dream and think about the future utility of this awesome microservice you will be building. You’ll be driving down the road in the USA and, using your mobile device, you’ll be able to invoke the microservice in the stdlib cloud and find the name of the city in your current location! You might even be driving in a convertible along the beach in San Diego. Enough dreaming! Let’s transition back to reality and get this microservice on the road. Pun intended? 🙂

Invoking the microservice from a URL#

The end game is that we want to create a microservice that can be invoked in a variety of ways including from a URL (HTTP GET). I have already released a sample of the microservice we will be building. The URL will look similar to this:

https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16

Go ahead and launch the microservice URL (https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16{.uri}) now since it is already deployed in the stdlib cloud. You should see the following results:

{
  "zipcode": "92101",
  "state_abbr": "CA",
  "latitude": "32.719601",
  "longitude": "-117.16246",
  "city": "San Diego",
  "state": "California",
  "distance": 0.2343729936809432
}

Feel free to tweak the URL parameters (lat and lon) and see how the results change.

Let’s examine the various components of this stdlib URL:

https://f.stdlib.com/<username>/gps/findcity?lat=32.72&lon=-117.16

• f.stdlib.com – this is the host name of the stdlib function-as-a-service platform, where all the magic takes place.
• – the stdlib username which is thisdavej in my case, and will be your username after you build your microservice.
• gps – the name of the microservice. You can think of this as a namespace for various functions.
• findcity – the name of a function created within the microservice
• lat/lon – URL parameters that are supplied to the function.

As a side note, my microservice (and yours too after you build it) can also be invoked via an HTTP GET in this form:

https://thisdavej.api.stdlib.com/gps?lat=32.72&lon=-117.16

You will notice that we can omit findcity since we will be configuring findcity as the default function that is invoked behind the scenes by the gps microservice if no function name is supplied. While it’s a personal decision, I prefer to use the full URL of https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16 since gps/findcity feels more self-documenting and I have the freedom to add other relevant functions under the gps namespace. We could simply create a findcity service and create a default function called main (or any other name) if we had no plans to create additional functions under gps.

Invoking the microservice from the command line#

Another cool feature of Polybit’s stdlib is that we can also invoke microservices from the terminal using the lib command. The lib command is installed as part of the npm global install of lib.cli which you already completed above. Let’s go ahead and use lib to invoke the microservice now:

lib thisdavej.gps.findcity --lat 32.72 --lon -117.16

Sure enough, you should see a JSON object returned containing “San Diego, CA” data.

Similar to the URL, lib can be invoked without specifically including the default function name of findcity as follows:

lib thisdavej.gps --lat 32.72 --lon -117.16

I’ve also employed some special tricks (explained further below) so we can invoke our microservice through lib in more of a terse fashion without specifying --lat and --lon like this:

lib thisdavej.gps.findcity 32.72 -117.16

We’ll discuss how this is done later, but I wanted you to see it in action up front.

Now that we have seen the end goal, follow along with me so you can create your first microservice!

Initialize your workspace#

First, let’s create a directory for our stdlib files and navigate to it:

mkdir stdlib-workspace
cd stdlib-workspace

The lib command, available on our system since we globally installed the lib npm package above, will prove to be our best friend as we create our microservice. Let’s start using it now to initialize our workspace:

lib init

The init command will prompt you to enter an e-mail address associated with your stdlib account. Assuming you don’t have an stdlib account, you will be given the opportunity to create an account with associated e-mail address and password now. You will need an account and need to be logged in so you can ultimately push your code to the stdlib cloud and run your microservice.

If you do not wish to create an stdlib account at this time, you can run lib init --no-login instead; however, you will be working in an offline mode and will not be able to deploy your service to the stdlib cloud.  This is not the recommended route since there’s not much benefit in creating microservices if they cannot be accessed in the cloud. 🙂  If you do start in this offline mode, you will ultimately need to run a lib login command when you are ready to push your changes (described later).  As part of the login process, you will be given an opportunity to create an stdlib account if you do not already have one.

Create your service#

To create your first service, issue the following command:

lib create

Enter the following information when prompted:

• Service Name: gps
• Default Function Name: findcity

There you go! You have successfully provisioned the barebones structure of your microservice. Let’s review this structure now.

A brief tour of the stdlib service directory structure#

Navigate to the directory of your newly created gps service (substituting<username> with the stdlib username you created above):

cd <username>/gps

In the gps directory, you will find the following structure:

- f/
  - gps/
    - function.json
    - index.js
- package.json
- env.json
- README.md

Here are some highlights of these components:

package.json#

This file utilizes the npm package.json format so it may look familiar to you:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

{
  "name": "gps",
  "version": "0.0.0",
  "description": "GPS Service - Functions to return results based on GPS coordinates",
  "author": "thisdavej <[email protected]>",
  "main": "f/findcity/index.js",
  "dependencies": {},
  "private": true,
  "stdlib": {
    "name": "thisdavej/gps",
    "defaultFunction": "findcity",
    "timeout": 10000,
    "publish": false
  }
}

Most of this information is autogenerated for you. Key fields include:

• name – The name your service will be published to. Usually in the format /.
• description – A description for your service. The description appears when/if you publish your service to the stdlib central registry (described momentarily). I changed line 4 (“description”) from the default value of “Service” to a more detailed description based on the service we are developing.
• defaultFunction – The default entry point for your service if no function is provided. Services are accessed by <username>/<service>/<function>. Our default function is findcity so <username>/gps will map to <username>/gps/findcity.
• publish – Set this value to true if you desire to publish your service to the stdlib central registry. (We’re not ready yet.) After pushing your service to the stdlib cloud, you will be able to use it right away; however, setting publish to true will enable your service to be viewable by others and accessible from the stdlib search page.

env.json#

The default env.json file looks like this:

{
  "dev": {
    "key": "value"
  },
  "release": {
    "key": "value"
  }
}

This file contains environment variables to be sent to the process.env variable. The values are used as follows:

• “dev” – used when you are either working locally or on the stdlib staging (dev) environment
• “release” – used when you push a release to stdlib

function.json#

This file is located under the f folder in a folder called findcity which is our default function:

{
  "name": "findcity",
  "description": "Function",
  "args": [
    "First argument",
    "Second argument"
  ],
  "kwargs": {
    "alpha": "Keyword argument alpha",
    "beta": "Keyword argument beta"
  },
  "http": {
    "headers": {}
  }
}

Here are some notes on the fields:

• “name”: The function name. Must match f/<function_path> exactly, or the registry will throw an error.
• “description”: A short description of the function.
• “args”: An array containing information about the arguments the function expects.
• “kwargs”: An object (key-value pairs) containing information about the keywords arguments the function expects.

index.js#

The code associated with our function.json file resides in the index.js file. Here are the default contents (sans comments):

module.exports = (params, callback) => {

  callback(null, 'hello world');

};

Install npm packages needed by our service#

We are now ready to move beyond the barebones structure and create our service. If you have not already there, go ahead and navigate to the directory of your newly created gps service (substituting <username> with the stdlib username you created above):

cd <username>/gps

As a first step, let’s install the excellent cities npm package from Steven Lu to help us accomplish the goal of getting a city name based on GPS coordinates.

npm install --save cities

We utilize the npm --save parameter so that the cities package is included as a dependency in our package.json file. This will be needed when we deploy our service to the cloud.

Write the function code#

Next, let’s write the code for our function. Go ahead an open the gps/f/findcity/index.js file and modify it as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

const cities = require('cities');

module.exports = (params, cb) => {

  const lat = !isNaN(params.args[0]) ? params.args[0] : params.kwargs.lat;
  const lon = !isNaN(params.args[1]) ? params.args[1] : params.kwargs.lon;

  let format = params.kwargs.fmt;
  if (typeof format !== 'undefined')
    format = format.trim().toLowerCase();

  if (lat === undefined || lon === undefined) {
    let message;
    if (lat === undefined && lon == undefined)
      message = 'Need to specify values for lat and lon';
    else if (lat == undefined)
      message = 'Need to specify values for lat';
    else
      message = 'Need to specify values for lon';
    cb(message);
    return;
  }

  const info = cities.gps_lookup(lat, lon);
  if (info.distance > 75) {
    cb('City lookup limited to GPS coordinates in the United States');
  } else {

    let result;
    if (format === 'simple')
      result = `${info.city}, ${info.state_abbr} ${info.zipcode}`;
    else
      result = info;
    cb(null, result);
  }
};

In line 1, we load the cities module and make it available for use in our function.

In line 3, we export our function which utilizes the stdlib function format consisting of the following arguments:

• params: an object containing the arguments (params.args) and keyword arguments (params.kwargs) passed to the function.
• callback: a callback that ends function execution, expecting an error argument (or null if no error) and a JSON-serializable result (or a Buffer for file processing).

In line 5, we utilize a special trick to retrieve a value for lat (latitude) from our function. Recall that our service can be invoked from the command line in two different ways:

lib thisdavej.gps --lat 32.72 --lon -117.16
lib thisdavej.gps.findcity 32.72 -117.16

In the first command, a value for latitude is supplied to params.kwargs.lat (a keyword argument). The second command supplies a value to params.arg[0] for the latitude. The longitude works in a similar fashion.

Consider how the microservice is invoked through a URL:

https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16

When accessing services via HTTP GET, all query parameters are converted to keyword arguments. Here, the lat URL parameter supplies a value to the params.kwargs.lat keyword argument. The args array is not available through a URL parameter, and thus we utilize the following code in line 5 to retrieve the value for lat from the kwargs (keyword arguments) since it is not available on the args array when invoked via HTTP GET.

const lat = !isNaN(params.args[0]) ? params.args[0] : params.kwargs.lat;

I do not recommend that you employ this trick on a regular basis to retrieve a value from the args parmeter and fall back to a kwargs parameter if the args parameter is not available. It it probably easier to simply use the kwargs parameters if you will be doing HTTP GET commands. I am primarily motivated to use it here to help teach you the nuances of the args parameters versus the kwargs parameters.

In line 8, we look for the presence of a kwargs parameter called fmt. We ultimately use this later in the function to determine the data that we return when the function is called.

In line 12, we confirm we have received values for latitude and longitude. If not, we invoke the callback (callback) to provide an error.

In line 24, we summon the cities.gps_lookup function to find the city based on the GPS coordinates. When this function is called, it returns a JSON object with the following structure (assuming the fmt kwargs parameter does not equal simple):

{
    "zipcode": "92101",
    "state_abbr": "CA",
    "latitude": "32.719601",
    "longitude": "-117.16246",
    "city": "San Diego",
    "state": "California",
    "distance": 0.2343729936809432
}

As shown, the object returned includes a distance value, and this value tells us how far the supplied GPS coordinates are from the center of the closest city. We are thus able to use this information in line 14 as a reality check to ensure that our city is within the United States.

If, however, we set the value of fmt to simple, and invoke the service through a URL, for example, of https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16&fmt=simple, the following data will be returned instead:

"San Diego, CA 92101"

Modify function.json#

Next, we modify function.json for our service:

{
  "name": "findcity",
  "description": "Find city based on GPS Coordinates",
  "kwargs": {
    "lat": "Latitude",
    "lon": "Longitude",
    "fmt": "Format of the data to return. If fmt='simple', just the city, state, and zip code are returned."
  },
  "http": {
    "headers": {}
  }
}

To provide documentation for future consumers of our function, we add a description and specify information about our kwargs (keyword arguments). We remove the args section since we are primarily using the keyword arguments and don’t want to confuse people by listing our latitude and longitude parameters twice.

Invoke our service locally#

We are ready to invoke our service locally so we can test it and refine it before pushing it to the stdlib cloud. Navigate to your service directory if you are not already there:

cd <username>/gps

We can invoke our function (index.js) as follows:

lib .

Since we do not supply values for latitude and longitude, we should see a “Need to specify values for lat and lon” error message. Perfect! It looks like our error checking code is working.

Now, let’s supply values for latitude and longitude through the args array:

lib . 32.72 -117.16

You should a result a JSON object returned now which with “San Diego, CA” data. It’s working!

Finally, invoke your function locally using the kwargs (keyword arguments):

lib . --lat 32.72 --lon -117.16

You should see the same JSON object with “San Diego, CA” data returned.

We can also invoke our function to return a simpler data structure by including the fmt keyword argument.

lib . --lat 32.72 --lon -117.16 --fmt simple

You should see the text “San Diego, CA 92101” returned rather than the more comprehensive JSON object.

The above calls to our service through lib utilize our default function. We can also achieve the same results by invoking lib like this:

lib .findcity --lat 32.72 --lon -117.16

Register our service in staging cloud#

We can register our service to either run in the staging (dev) cloud or the production (release) cloud. Staging environments are mutable and can be overwritten. Releases are immutable and cannot be overwritten; however, they can be torn down.

To register our service in the staging (dev) cloud, issue the following command from our service directory (gps):

lib up dev

Let’s give it a test! First, let’ invoke our cloud-based service through the command line using lib where <username> is your user name:

lib <username>.gps[@dev].findcity 32.72 -117.16

Note that we include [@dev] to invoke our service in the staging cloud. You should see our same JSON object result returned.

Next, let’s invoke our service from the web using the following URL:

https://f.stdlib.com/<username>/gps@dev/findcity?lat=32.72&lon=-117.16

Note: the name of dev for staging is arbitrary. You could use a different name such as test to create a different staging environment, provided that you add @test to your service name when invoking it.

Register our service in the production cloud#

To register our service as a release in production, issue the following command from the service directory (gps):

lib release

Let’s see it in action! First, let’ invoke it through the command line using lib where <username> is your user name:

lib <username>.gps.findcity 32.72 -117.16

Next, we can invoke our service from the web using the following URL:

https://f.stdlib.com/<username>/gps/findcity?lat=32.72&lon=-117.16

We are in business and our microservice is in production!

Publish our service to the registry#

After you are satisfied with the service you have created, you can (if you choose) publish your service and make it publicly searchable on the stdlib registry.

Update README.md file#

As a first step, modify the markdown in the README.md file located in the root of your service directory since the contents of this file will be used to provide the primary documentation for your published service with the help of the stdlib markdown service which will convert the markdown to HTML behind the scenes.

Side note: I have written a popular tutorial for building a markdown editor using Visual Studio Code that can help you write markdown if you are using VS Code as your editor.

The stdlib markdown service supports both standard markdown and GFM (GitHub Flavored Markdown), and supports syntax highlighting of fenced code blocks with the help of highlight.js.  You will find a list of highlight.js languages and aliases that can be used for syntax highlighting fenced code blocks here.  Fenced code blocks are denoted with three back ticks at the start and end of each code block as shown in the following JSON syntax example:

```JSON
{
  "zipcode": "92101",
  "state_abbr": "CA",
  "latitude": "32.719601",
  "longitude": "-117.16246",
  "city": "San Diego",
  "state": "California",
  "distance": 0.2343729936809432
}

Go ahead and update your `README.md` file.  Here is a complete sample for the `gps` service that you can use as a starting point:

```md
# GPS Service

This service provides functions to return results based on GPS coordinates.  There is currently one function available:

## findcity - Find the name of city in the USA based on GPS coordinates

### Parameters

#### Keyword arguments

- lat: latitude (the angular distance of a place north or south of the earth's equator)
- lon: longitude (the angular distance of a place east or west of the meridian at Greenwich, England)
- fmt: format of the data to return.  If fmt = `simple`, just the city, state, and zip code are returned
rather than a full JSON object.

### Usage

#### Command line

Return a JSON object with city information based on latitude and longitude

```bash
$ lib thisdavej.gps.findcity --lat 32.72 --lon -117.16

This returns the following JSON object:

{
  "zipcode": "92101",
  "state_abbr": "CA",
  "latitude": "32.719601",
  "longitude": "-117.16246",
  "city": "San Diego",
  "state": "California",
  "distance": 0.2343729936809432
}

Return a more concise string representing the city:

lib thisdavej.gps.findcity --lat 32.72 --lon -117.16 --fmt simple

This returns the following simple string:

"San Diego, CA 92101"

HTTP GET#

Return a JSON object with city information based on latitude and longitude

https://thisdavej.api.stdlib.com/gps/findcity?lat=32.72&lon=-117.16

Web and Node.js#

const f = require('f');

f('thisdavej.gps.findcity')({lat: 32.72, lon: -117.16}, (err, response) => {
  // handle error or response
});

Bugs/limitations#

This function is currenly only capable of finding cities in the USA.

Author#

Dave Johnson (https://thisdavej.com/)

You can visit the [thisdavej/gps service page][30] to see this `README.md` page in action.

#### Publish to stdlib registry

Excellent – you have modified the `README.md` file to provide documentation for the users of your service.  We are now ready to publish!  To accomplish this goal, modify `package.json`:

  * set `"publish"` to a value of `true`.
  * increment the version number in the `"version"` field. This is necessary since we cannot overwrite a version that is already released in production. Follow semantic versioning [semvar][31] guidelines (`MAJOR.MINOR.PATCH`) when specifying a new version for your service. For example, increment the major version if you are introducing breaking changes to the API surface of your microservice.

Lastly, release the new version of the service into production from the terminal:

```shell
$ lib release

Removing a service#

Use the following command to remove a service where <environment> will be dev or release:

lib down <environment>

To remove a specific version, use this command:

lib down -r <version>

You can also use lib rollback to remove the currently specified release if it was published by accident.

Restarting a service#

Utilize the following command syntax to restart your microservice if it stops working in the cloud:

lib restart [<environment>] | [-r <version>]

Notes:

• -r specified the release (production environment)
• Non-release (mutable) environments (e.g. dev) do not have versions associated with them.

Here are some examples:

lib restart dev # restart dev environment (there are no versions associated with dev/staging environents.)
lib restart -r # restart release envivonment
lib restart -r 0.0.2 # release release environment 0.0.2

This shouldn’t be necessary, but the option exists should you encounter any errors.

Rebuilding a service#

You are also able to rebuild a service. This reinstalls package dependencies and uses the most up-to-date version of the stdlib microservice software. This may be encouraged as Polybit rolls out stdlib updates aimed at increasing performance or enhancing security. Here is the syntax for rebuiding your service:

lib rebuild [<environment>] | [-r <version>]

Here are some examples:

lib rebuild dev # rebuild dev environment (there are no versions associated with dev/staging environents.)
lib rebuild -r # rebuild release envivonment
lib rebuild -r 0.0.2 # rebuild release environment 0.0.2

Creating additional service functions#

To create additional endpoints (functions) for your service, navigate to the directory of your service (substituting <username> with your stdlib username). For example:

cd <username>/gps

Next, utilize the following command syntax:

lib&nbsp;f:create <function>

To create a function called navigate, for example, you would invoke the following command:

lib f:create navigate

This will create a new navigate function that you can modify to create your next awesome function for the world. 🙂   Since navigate is not the default function, it will be invoked as follows:

lib .navigate

You are equipped and ready to create all kinds of amazing microservices and functions!

Conclusion#

We successfully leveraged Polybit’s stdlib platform to publish a microservice for retrieving the name of a city based on GPS coordinates! The process was amazingly simple and we were able to focus on writing code rather than stressing over infrastructure or getting immersed in various other yak shaving diversions.

In our next article, we will learn how to consume stdlib microservices through a wide variety of methods. It will be another great time of learning and fun!

Follow @thisDaveJ (Dave Johnson) on X to stay up to date with the latest tutorials and tech articles.