Code with Hugo: ES6 by example: a module/CLI to wait for Postgres

                            July 9, 2018

                Code with Hugo: ES6 by example: a module/CLI to wait for Postgres

                        ES6 by example: a module/CLI to wait for Postgres
When using docker-compose, it’s good practice to make anything that relies on Postgres wait for it to be up before launching. This avoids connection issues inside the app.
This post walks through how to deliver this functionality both as a CLI and a module that works both as a CommonJS module (require) and ES modules, without transpilation.
“A fast, production ready, zero-dependency ES module loader for Node 6+!” is esm’s promise. From this sample project, it’s worked.
Writing ES modules without a build step 🎨
To begin we install esm: npm install --save esm.
Next we’ll need a file for our module, wait-for-pg.js:
export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

Trying to run this file with Node will throw:
$ node wait-for-pg.js
/wait-for-pg/wait-for-pg.js:1
export const DEFAULT_MAX_ATTEMPTS = 10;
^^^^^^

SyntaxError: Unexpected token export

export and import don’t work in Node yet (without flags), the following runs though:
$ node -r esm wait-for-pg.js

That’s if we want to run it as a script, say we want to let someone else consume it via require we’ll need an index.js with the following content:
require = require('esm')(module);
module.exports = require('./wait-for-pg');

We can now run index.js as a script:
$ node index.js

We can also require it:
$ node # start the Node REPL
> require('./index.js')
{ DEFAULT_MAX_ATTEMPTS: 10,
  DEFAULT_DELAY: 1000 }

To tell users wanting to require the package with Node, we can use the "main"  field in package.json:
{
  "main": "index.js",
  "dependencies": {
    "esm": "^3.0.62"
  }
}

Sane defaults 🗃
To default  databaseUrl, maxAttempts and delay, we use ES6 default parameters + parameter destructuring.
Let’s have a look through some gotchas of default parameters that we’ll want to avoid:

Attempting to destructure ‘null’ or ‘undefined’
‘null’ remains, undefined gets defaulted

Attempting to destructure null or undefined 0️⃣
export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

Callings the following will throw:
$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)
> waitForPostgres(null)
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)

To avoid this, we should add = {} to default the parameter that’s being destructured (wait-for-pg.js):
export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

It now runs:
$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
postgres://postgres@localhost 10 1000

The values were successfully defaulted when not passed a parameter. However the following still errors:
> waitForPostgres(null)
postgres://postgres@localhost 10 1000
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)

‘null’ remains, undefined gets defaulted 🔎
$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
null 10 1000

The values explicitly set as null doesn’t get defaulted whereas an explicit undefined and an implicit one do, that’s just how default parameters work, which isn’t exactly like the old-school way of writing this:
export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres(options) {
  const databaseUrl = (
    options && options.databaseUrl ||
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  );
  const maxAttempts = options && options.maxAttempts || DEFAULT_MAX_ATTEMPTS;
  const delay = options && options.delay || DEFAULT_DELAY;
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}

Which would yield the following:
$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
'postgres://postgres@localhost' 10 1000

Since null is just as falsy as undefined 🙂 .
Waiting for Postgres with async/await 🛎
Time to implement wait-for-pg.
To wait for Postgres we’ll want to:

try to connect to it
if that fails
try again later
if that succeeds
finish

Let’s install a Postgres client, pg using: npm install --save pg
pg has a Client object that we can pass a database URL to when instantiating it (new Client(databaseUrl)). That client instance has a .connect method that returns a Promise which resolves on connection success and rejects otherwise.
That means if we mark the waitForPostgres function as async, we can await the .connect call. 
When await-ing a Promise, a rejection will throw an error so we wrap all the logic in a try/catch.

If the client connection succeeds we flip the loop condition so that the function terminates
If the client connection fails
we increment the retries counter, if it’s above the maximum number of retries (maxAttempts), we throw which, since we’re in an async function throw is the equivalent of doing Promise.reject
otherwise we call another function that returns a Promise (timeout) which allows us to wait before doing another iteration of the loop body
We make sure to export function waitForPostgres() {}

wait-for-pg.js:
import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export async function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  let didConnect = false;
  let retries = 0;
  while (!didConnect) {
    try {
      const client = new Client(databaseUrl);
      await client.connect();
      console.log('Postgres is up');
      client.end();
      didConnect = true;
    } catch (error) {
      retries++;
      if (retries > maxAttempts) {
        throw error;
      }
      console.log('Postgres is unavailable - sleeping');
      await timeout(delay);
    }
  }
}

Integrating as a CLI with meow 😼
meow is a CLI app helper from Sindre Sohrus, install it: npm install --save meow
Create wait-for-pg-cli.module.js:
import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
console.log(cli.input, cli.flags);

We use inferType so that the values for maxAttempts and delay get converted to numbers instead of being strings.
We can run it using:
$ node -r esm wait-for-pg-cli.module.js
[] {}

The following is a template string, it will replace things inside of ${} with the value in the corresponding expression (in this instance the value of the DEFAULT_MAX_ATTEMPTS and DEFAULT_DELAY variables)
`
  Usage
    $ wait-for-pg <DATABASE_URL>
  Options
    --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
    --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
  Examples
    $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
    # waits for postgres, 5 attempts at a 3s interval, if
    # postgres becomes available, run 'npm start'
`;

To get the flags and first input, wait-for-pg-cli.module.js:
import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);

If you don’t have a Postgres instance running on localhost the following shouldn’t print Here, that’s thanks to process.exit(1) in the .catch block:
$ node -r esm wait-for-pg-cli.module.js -c 5 && echo "Here"
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping

Packaging and clean up 📤
We can use the "bin" key in package.json to be able to run the command easily:
{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  }
}

Where wait-for-pg-cli.js is:
#!/usr/bin/env node
require = require("esm")(module/*, options*/);
module.exports = require('./wait-for-pg-cli.module');

Don’t forget to run chmod +x wait-for-pg-cli.js
esm allows us to use top-level await, that means in wait-for-pg-cli.module.js, we can replace:
waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);

With:
try {
  await waitForPostgres({
    databaseUrl: cli.input[0],
    maxAttempts: cli.flags.maxAttempts,
    delay: cli.flags.delay,
  });
  process.exit(0);
} catch (error) {
  process.exit(1);
}

Running the CLI throws:
$ ./wait-for-pg-cli.js
wait-for-pg/wait-for-pg-cli.module.js:36
  await waitForPostgres({
  ^^^^^

SyntaxError: await is only valid in async function

We need to add "esm" with "await": true in package.json:
{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "esm": {
    "await": true
  }
}

This now works:
$ ./wait-for-pg-cli.js -c 1
Postgres is unavailable - sleeping

Extras
Publishing to npm with np

Run: npm install --save-dev np
Make sure you have a valid "name" field in package.json, eg. "@hugodf/wait-for-pg"
npx np for npm v5+ or ./node_modules/.bin/np (npm v4 and down)

Pointing to the ESM version of the module
Use the "module" fields in package.json
{
  "name": "wait-for-pg",
  "version": "1.0.0",
  "description": "Wait for postgres",
  "main": "index.js",
  "module": "wait-for-pg.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "devDependencies": {
    "np": "^3.0.4"
  },
  "esm": {
    "await": true
  }
}

A Promise wait-for-pg implementation
import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY,
} = {},
  retries = 1
) {
  const client = new Client(databaseUrl);
  return client.connect().then(
    () => {
      console.log('Postgres is up');
      return client.end();
    },
    () => {
      if (retries > maxAttempts) {
        return Promise.reject(error);
      }
      console.log('Postgres is unavailable - sleeping');
      return timeout(delay).then(
        () => waitForPostgres(
          { databaseUrl, maxAttempts, delay },
          retries + 1
        )
      );
    }
  );
}

                            Don't miss what's next. Subscribe to Code with Hugo: JS & Node.js in Prod: