Migrate Sequelize files

What has changed from previous versions

In version 8 we introduced the application-wide scopes and dynamic smart action form along with hooks on bulk and global smart actions.

Since version 7, we introduced a new structure of our generated projects to ease the support of multiple databases.

This new structure is the major change we need to consider to correctly perform the migration of your files. The idea of this document is to guide you through the project architecture and make as few changes as possible to benefit from TypeScript.

The key concepts to handle multi databases are the following:

• A configuration file to export your connection options to your databases.

• A dynamic loading mechanism to dynamically load your models depending on their database connections.

The files related to these key concepts are the files under the /models and the /config directories. Let's handles these files first.

Define your models

Goals:

• Create models class and interfaces to code using TypeScript

• Synchronize models with the database

To migrate our JavaScript models to TypeScript, we will take the following model generated by lumber as an example:

models/articles.js

module.exports = (sequelize, DataTypes) => {

  const { Sequelize } = sequelize;
  
  const Articles = sequelize.define('articles', {
    title: {
      type: DataTypes.STRING,
    },
    body: {
      type: DataTypes.STRING,
    },
  }, {
    tableName: 'articles',
    underscored: true,
    schema: process.env.DATABASE_SCHEMA,
  });

  Articles.associate = (models) => {
    Articles.belongsTo(models.owners, {
      foreignKey: {
        name: 'ownerIdKey',
        field: 'owner_id',
      },
      as: 'owner',
    });
  };

  return Articles;
};

To declare our models using TypeScript we need to follow the documentation from Sequelize. Two interfaces need to be created describing the attributes for the creation and existing records, and a model class should use these interfaces to benefit from TypeScript:

models/articles.ts

import { Model, ModelCtor } from 'sequelize';

interface IArticleAttributes {
  id: number;
  title: string | null;
  body: string | null;
  createdAt: Date;
  updatedAt: Date;
}

interface IArticleCreationAttributes {
  title: string | null;
  body: string | null;
}

export class Article extends Model<IArticleAttributes, IArticleCreationAttributes> {
  public id!: number;
  public title!: string | null;
  public body!: string | null;
  public createdAt!: Date;
  public updatedAt!: Date;

  public static associate(models: Record<string, ModelCtor<Model>>): void {
    Article.belongsTo(models.owners, {
      foreignKey: {
        name: 'ownerIdKey',
        field: 'owner_id',
      },
      as: 'owner',
    })
  }
}

Note the static associate function. This function is mandatory to perform dynamic loading and associate models with each other, we will dive into that later on.

Now our TypeScript definitions are done, we still need to initialize our interfaces and class with the articles table in the database using Sequelize:

models/articles.ts

import { Model, Sequelize, DataTypes, ModelCtor } from 'sequelize';

... //previous class and interfaces are hidden to ease reading

export default function(sequelize: Sequelize, dataTypes: typeof DataTypes): typeof Model {

  Article.init({
    id: {
      type: dataTypes.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
    title: {
      type: new dataTypes.STRING(128),
      allowNull: true,
    },
    body: {
      type: new dataTypes.STRING(128),
      allowNull: true,
    },
  },{
    tableName: 'articles',
    underscored: true,
    modelName: 'articles',
    sequelize,
  });

  return Article;
}

Note that the function to initialize the model is the default export from the model file. This is mandatory and will be explained in the next section.

And that's it! Your model definition is finished and you will be able to use it in your code soon, just go ahead to the next section to see how to instantiate your models.

Connect to your databases

Goals:

• Connect to each of your databases

• Import models definition in those connections

The database connections are configured in config/databases.js. Here is an example of a fresh new generated project with one connection:

config/databases.js

const path = require('path');

const databaseOptions = {
  logging: !process.env.NODE_ENV || process.env.NODE_ENV === 'development' ? console.log : false,
  pool: { maxConnections: 10, minConnections: 1 },
  dialectOptions: {},
};

if (process.env.DATABASE_SSL && JSON.parse(process.env.DATABASE_SSL.toLowerCase())) {
  const rejectUnauthorized = process.env.DATABASE_REJECT_UNAUTHORIZED;
  if (rejectUnauthorized && (JSON.parse(rejectUnauthorized.toLowerCase()) === false)) {
    databaseOptions.dialectOptions.ssl = { rejectUnauthorized: false };
  } else {
    databaseOptions.dialectOptions.ssl = true;
  }
}

module.exports = [{
  name: 'default',
  modelsDir: path.resolve(__dirname, '../models'),
  connection: {
    url: process.env.DATABASE_URL,
    options: { ...databaseOptions },
  },
}];

This file is responsible for providing the database connections with the corresponding options. In the end, this file should export an array of connections. If we translate this file into TypeScript, we have to export the same database structure, here is an example:

config/databases.ts

import * as path from 'path';
import { Options } from "sequelize";
import { DatabaseConfiguration } from "forest-express-sequelize";

const databaseOptions: Options = {
  logging: !process.env.NODE_ENV || process.env.NODE_ENV === 'development' ? console.log : false,
  pool: { max: 10, min: 1 },
  dialectOptions: { },
};

if (process.env.DATABASE_SSL && JSON.parse(process.env.DATABASE_SSL.toLowerCase())) {
  const rejectUnauthorized = process.env.DATABASE_REJECT_UNAUTHORIZED;
  if (rejectUnauthorized && (JSON.parse(rejectUnauthorized.toLowerCase()) === false)) {
    databaseOptions.dialectOptions["ssl"] = { rejectUnauthorized: false };
  } else {
    databaseOptions.dialectOptions["ssl"] = true;
  }
}

const databasesConfiguration: DatabaseConfiguration[] = [{
  name: 'default',
  modelsDir: path.resolve(__dirname, '../models'),
  connection: {
    url: process.env.DATABASE_URL,
    options: { ...databaseOptions },
  },
}];

export default databasesConfiguration;

Now your connections are configured, let's load the models for every connection.

A generated project uses a dynamic model instantiation approach. To do so, the models/index.js file under the models folder is responsible for browsing your connections' configuration and models to load them. Here is an example of this generated file:

models/index.js

const fs = require('fs');
const path = require('path');
const Sequelize = require('sequelize');

const databasesConfiguration = require('../config/databases');

const connections = {};
const db = {};

databasesConfiguration.forEach((databaseInfo) => {
  const connection = new Sequelize(databaseInfo.connection.url, databaseInfo.connection.options);
  connections[databaseInfo.name] = connection;

  const modelsDir = databaseInfo.modelsDir || path.join(__dirname, databaseInfo.name);
  fs
    .readdirSync(modelsDir)
    .filter((file) => file.indexOf('.') !== 0 && file !== 'index.js')
    .forEach((file) => {
      try {
        const model = connection.import(path.join(modelsDir, file));
        db[model.name] = model;
      } catch (error) {
        console.error(`Model creation error: ${error}`);
      }
    });
});

Object.keys(db).forEach((modelName) => {
  if ('associate' in db[modelName]) {
    db[modelName].associate(db);
  }
});

db.objectMapping = Sequelize;
db.connections = connections;

module.exports = db;

Note that line 20 is the place where we should use the default export from our model files. At this line, this default export should be called and Sequelize will automatically detect that it is an initialization function, and will instantiate our Article class for example. Also, note that the association between models is done from line 28 to line 32. We should call the associate function we kept on our models at this same place. Otherwise, our models will not be linked to each other.

Finally, we still need to export at least Sequelize as objectMapping and the connections, the same way it is done in JavaScript.

Here is an implementation of all those requirements in TypeScript:

Please read line 16 carefully and observe that we also filter*.js.map files. If you do use mapping files you need to skip them as in this example, to not consider them as model files to import.

models/index.ts

import { readdirSync } from "fs";
import { join } from 'path';
import * as Sequelize from 'sequelize';
import databasesConfiguration from "../config/databases";

const connections: Record<string, Sequelize.Sequelize> = {};
const objectMapping = Sequelize;
const models: Record<string, typeof Sequelize.Model> = {};

databasesConfiguration.forEach((databaseInfo) => {
  const connection = new Sequelize.Sequelize(databaseInfo.connection.url, databaseInfo.connection.options);
  connections[databaseInfo.name] = connection;

  const modelsDir = databaseInfo.modelsDir || join(__dirname, databaseInfo.name);
  readdirSync(modelsDir)
    .filter((file) => file.indexOf('.') !== 0 && file !== 'index.js' && !file.includes('.map'))
    .forEach((file) => {
      try {
        const model = connection.import(join(modelsDir, file));
        models[model.name] = model;
      } catch (error) {
        console.error(`Model creation error: ${error}`);
      }
    });
});

Object.keys(models).forEach((modelName) => {
  if ('associate' in models[modelName]) {
    // @ts-ignore
    models[modelName].associate(models);
  }
});

export { objectMapping, models, connections };

Now we configured everything regarding connections and models, we can benefit from TypeScript and our models using:

const { Article } from '../models/article';

...

Artcile.findByPk(...)...

Initialize your Liana

The middlewares/forestadmin.js is pretty straightforward and just needs to be translated from .js to .ts . See the following snippet:

middlewares/forestadmin.ts

import * as chalk from 'chalk';
import { join } from 'path';
import { init, LianaOptions } from "forest-express-sequelize";
import { objectMapping, connections } from '../models';
import { Application } from "express";

export = async function forestadmin(app: Application): Promise<void> {
  const lianaOptions: LianaOptions = {
    configDir: join(__dirname, '../forest'),
    envSecret: process.env.FOREST_ENV_SECRET,
    authSecret: process.env.FOREST_AUTH_SECRET,
    objectMapping,
    connections,
  }

  app.use(await init(lianaOptions));

  console.log(chalk.cyan('Your admin panel is available here: https://app.forestadmin.com/projects'));

  return;
};

Override your routes

Nothing special needs to be done in your routes. Change the files extension to .ts , change the way dependencies are imported, and let yourself be guided by the Types we provide. Here is an example for the route to get an Article:

routes/articles.ts

import * as express from 'express';
import { PermissionMiddlewareCreator, RecordsGetter } from "forest-express-sequelize";
import { Article } from "../models/article";

...

router.get('/articles', permissionMiddlewareCreator.list(), async (request: ForestRequest, response) => {
  const { user, query } = request;
  const recordsGetter = new RecordsGetter(Article, user, query);
  const articles = await recordsGetter.getAll(request.params);
  const articlesSerialized = await recordsGetter.serialize(articles);
  response.json(articlesSerialized);
});

Note the request: ForestRequest to have access to user and query

Customize your collections

As for the previous sections, nothing special needs to be changed to your collection configuration. Change the files extension to .ts , change the way dependencies are imported, and let yourself be guided by the Types we provide. Here is an example for the Article collection:

forest/articles.ts

import { collection } from "forest-express-sequelize";

collection('articles', {
  actions: [],
  fields: [{
    field: 'Description',
    type: 'String',
  }],
  segments: [],
});