Migrate Mongoose files

What has changed from previous versions

In version 8 we introduced the application-wide scopes and dynamic smart action form along with hooks on bulk and global smart actions.

Since version 7, we introduced a new structure of our generated projects to ease the support of multiple databases. This new structure is the major change we need to consider to correctly perform the migration of your files.

The idea of this document is to guide you through the project architecture and make as few changes as possible to benefit from TypeScript.

The key concepts to handle multi databases are the following:

• A configuration file to export your connection options to your databases.

• A dynamic loading mechanism to dynamically load your models depending on their database connections.

The files related to these key concepts are the files under the /models and the /config directories. Let's handles these files first.

Define your models

What to achieve:

• Export an interface representing your model in TypeScript

• Export a mongoose Schema corresponding to your interface

To migrate our JavaScript models to TypeScript, we will take the following model generated by lumber as an example:

models/client.js

module.exports = (mongoose, Mongoose) => {

  const schema = Mongoose.Schema({
    'status': String,
    'firstname': String,
    ...
    'lastname': String,
    }]
  }, {
    timestamps: false,
  });

  return mongoose.model('clients', schema, 'clients');
};

As you can see, it has been generated to be loaded dynamically. This file actually exports a function accepting a mongoose connection as an argument, to link the model to it. This will not be required anymore.

To make this model work using TypeScript, we first need to declare an interface representing data we can encounter in it. We also need to export the mongoose Schema that describes the same data from the interface. Mongoose can only work with schemas, but TypeScript can't correctly infer types from this kind of structure. This is why we need both the interface for coding and the schema for mongoose.

models/client.ts

import { Schema, Document } from "mongoose";

export interface IClient {
  status: string,
  firstname: string,
  ...
  lastname: string,
}

const ClientSchema = new Schema<IClient>({
  status: { type: String, required: true },
  firstname: { type: String, required: true },
  ...
  lastname: { type: String, required: true },
}, {
  timestamps: false,
});

export ClientSchema;

Do note the usage of the interface in the Schema creation (line 10), this is how we link the TypeScript world to the mongoose world. Also, note that the model is not created in this file anymore, this will be done in the models/index.ts we are about to change in the next section.

Connect to your databases

Goals:

• Connect to each of your databases

• Import models definition in those connections

The database connections are configured in config/databases.js. Here is an example of a fresh new generated project with one connection:

config/databases.js

const path = require('path');

const databaseOptions = {
  useNewUrlParser: true,
  useUnifiedTopology: true,
};

module.exports = [{
  name: 'default',
  modelsDir: path.resolve(__dirname, '../models'),
  connection: {
    url: process.env.DATABASE_URL,
    options: { ...databaseOptions },
  },
}];

This file is responsible for providing the database connections with the corresponding options. In the end, this file should export an array of connections. If we translate this file into TypeScript, we have to export the same database structure, here is an example:

config/databases.ts

import { ConnectionOptions } from "mongoose";
import { DatabaseConfiguration } from "forest-express-mongoose";
import { resolve } from 'path';

const databaseOptions: ConnectionOptions = {
  useNewUrlParser: true,
  useUnifiedTopology: true,
};

const databasesConfiguration: DatabaseConfiguration[] = [{
  name: 'default',
  modelsDir: resolve(__dirname, '../models'),
  connection: {
    url: process.env.DATABASE_URL,
    options: { ...databaseOptions },
  },
}];

export  databasesConfiguration;

Now your connections are configured, let's load the models for every connection.

A generated project uses a dynamic model instantiation approach. To do so, the index.js file under the models folder is responsible for browsing your models and for loading them. Then, the file exports the models and you can access them from anywhere using const { client } = require('./models');. This works pretty well in JavaScript because you are not forced to strictly type what models exports. However using TypeScript, this kind of approach is not ideal because the dynamic part of this approach can not be typed, and thus, TypeScript will raise errors stating that models has no exported member client . Let's break down the index.js file step by step to understand what is going on inside.

models/index.js

const fs = require('fs');
const path = require('path');
const Mongoose = require('mongoose');

const databasesConfiguration = require('../config/databases');

const connections = {};
const db = {};

databasesConfiguration.forEach((databaseInfo) => {
  const connection = Mongoose.createConnection(databaseInfo.connection.url, databaseInfo.connection.options);
  connections[databaseInfo.name] = connection;

  const modelsDir = databaseInfo.modelsDir || path.join(__dirname, databaseInfo.name);
  fs
    .readdirSync(modelsDir)
    .filter((file) => file.indexOf('.') !== 0 && file !== 'index.js')
    .forEach((file) => {
      try {
        const model = require(path.join(modelsDir, file))(connection, Mongoose);
        db[model.modelName] = model;
      } catch (error) {
        console.error(`Model creation error: ${error}`);
      }
    });
});

db.objectMapping = Mongoose;
db.connections = connections;

module.exports = db;

Indexing models by their name (line 21) is the concept we want to avoid to benefit from TypeScript and code completion. Because it is dynamic at run time, TypeScript can't infer the structure of the exported models and thus, fails at compilation time.

For every connection you configured, this code will create the connection to the database and dynamically load the associated models (from line 10 to line 14). We have no choice but to break this dynamic approach and load the models by hand to be able to export them later on. The new static approach will allow TypeScript to infer the type of the exported object, and thus, will provide us with clean code completion.

models/index.ts

import * as Mongoose from 'mongoose';
import * as databasesConfiguration from "../config/databases";
import { ClientSchema, IClient } from '../models/clients';

const connections: Record<string, Mongoose.Connection> = {};
const objectMapping = Mongoose;

const connection = Mongoose.createConnection(databasesConfiguration[0].connection.url, databasesConfiguration[0].connection.options);
connections[connection.name] = connection;

const Client = connection.model<IClient>('clients', ClientSchema, 'clients');

export {
  objectMapping,
  connections,
  Client,
};

If you still want to work with such a dynamic loading mechanism, please ensure that either you don't generate .map.js files, or ensure you skip these files in the filter in the original index.jssnippet. Otherwise, you will encounter an error at runtime such as Model creation error: SyntaxError: Unexpected token ':'

Do note that we still export connections and Mongoose as objectMapping . This is still mandatory to allow your app to be correctly initialized.

This might look tedious at first, but in the end, you are now able to use your models with code completion like the following:

routes/clients.ts

import { Client } from '../models'

...

Client.findById(..)..

Initialize your Liana

The middlewares/forestadmin.js is pretty straightforward and just needs to be translated from .js to .ts . See the following snippet:

middlewares/forestadmin.ts

import * as chalk from 'chalk';
import { join } from 'path';
import { init, LianaOptions } from "forest-express-mongoose";
import { objectMapping, connections } from '../models';
import { Application } from "express";

export = async function forestadmin(app: Application): Promise<void> {
  const lianaOptions: LianaOptions = {
    configDir: join(__dirname, '../forest'),
    envSecret: process.env.FOREST_ENV_SECRET,
    authSecret: process.env.FOREST_AUTH_SECRET,
    objectMapping,
    connections,
  }

  app.use(await init(lianaOptions));

  console.log(chalk.cyan('Your admin panel is available here: https://app.forestadmin.com/projects'));

  return;
};

Override your routes

For your routes' code, nothing, in particular, should be changed in terms of structure. Just translate your JavaScript code into TypeScript.

To benefit from code completion, just ensure to use your models' interface when you manipulate your records. Here is an example with the client model we have been using in this documentation:

routes/clients.ts

import {
  ForestRequest,
  ...
  RecordGetter,
} from 'forest-express-mongoose';
import { Client } from '../models';
import { IClient } from '../models/clients';

...

router.get('/clients/:recordId', permissionMiddlewareCreator.details(), async (request: ForestRequest, response) => {
  const { user, query } = request;
  const clientGetter = new RecordGetter<IClient>(Client, user, query);
  const client = await clientGetter.get(request.params.recordId);
  const clientSerialized = await clientGetter.serialize(client);
  response.json(clientSerialized);
});

Note the request: ForestRequest to have access to user and query

After line 8, the type of the client record should be well inferred and your IDE should provide some code completion, related to the IClient interface. This is because we passed the interface to RecordGetter as a generic type.

Customize your collections

For your collections' code, nothing, in particular, should be changed in terms of structure.

The difference you will feel now is the code completion provided by your IDE, in conjunction with our Types. For example, If you start to create a smart field, you should be guided and a list of possible attributes to configure should be provided.