Create a Smart Collection with Amazon S3

Please be sure of your agent type and version and pick the right documentation accordingly.

Node.js

This is the documentation of the forest-express-sequelize and forest-express-mongoose Node.js agents that will soon reach end-of-support.

forest-express-sequelize v9 and forest-express-mongoose v9 are replaced by @forestadmin/agent v1.

Please check your agent type and version and read on or switch to the right documentation.

Ruby on Rails

This is still the latest Ruby on Rails documentation of the forest_liana agent, you’re at the right place, please read on.

Python

This is the documentation of the django-forestadmin Django agent that will soon reach end-of-support.

If you’re using a Django agent, notice that django-forestadmin v1 is replaced by forestadmin-agent-django v1.

If you’re using a Flask agent, go to the forestadmin-agent-flask v1 documentation.

Please check your agent type and version and read on or switch to the right documentation.

PHP

This is the documentation of the forestadmin/laravel-forestadmin Laravel agent that will soon reach end-of-support.

If you’re using a Laravel agent, notice that forestadmin/laravel-forestadmin v1 is replaced by forestadmin/laravel-forestadmin v3.

If you’re using a Symfony agent, go to the forestadmin/symfony-forestadmin v1 documentation.

Please check your agent type and version and read on or switch to the right documentation.

Create a Smart Collection with Amazon S3

Creating the Smart Collection

On our Live Demo, we’ve stored the Legal Documents of a Company on Amazon S3. In the following example, we show you how to create the Smart Collection to see and manipulate them in your Forest admin.

SQL

First, we declare the legal_docs collection in the forest/ directory. In this Smart Collection, all fields are related to S3 attributes except the field is_verified that is stored on our database in the collection documents.

You can check out the list of available field options if you need it.

You MUST declare an id field when creating a Smart Collection. The value of this field for each record MUST be unique. On the following example, we simply generate a random UUID.

/forest/legal_docs.js

const { collection } = require('forest-express-sequelize');
const models = require('../models');

collection('legal_docs', {
  fields: [
    {
      field: 'id',
      type: 'String',
    },
    {
      field: 'url',
      type: 'String',
      widget: 'link',
      isReadOnly: true,
    },
    {
      field: 'last_modified',
      type: 'Date',
      isReadOnly: true,
    },
    {
      field: 'size',
      type: 'String',
      isReadOnly: true,
    },
    {
      field: 'is_verified',
      type: 'Boolean',
      isReadOnly: false,
    },
  ],
});

Mongodb

First, we declare the legal_docs collection in the forest/ directory. In this Smart Collection, all fields are related to S3 attributes except the field is_verified that is stored on our database in the collection documents.

You can check out the list of available field options here if you need it.

You MUST declare an id field when creating a Smart Collection. The value of this field for each record MUST be unique. On the following example, we simply generate a random UUID.

/forest/legal_docs.js

const { collection } = require('forest-express-mongoose');
const models = require('../models');

collection('legal_docs', {
  fields: [
    {
      field: 'id',
      type: 'String',
    },
    {
      field: 'url',
      type: 'String',
      widget: 'link',
      isReadOnly: true,
    },
    {
      field: 'last_modified',
      type: 'Date',
      isReadOnly: true,
    },
    {
      field: 'size',
      type: 'String',
      isReadOnly: true,
    },
    {
      field: 'is_verified',
      type: 'Boolean',
      isReadOnly: false,
    },
  ],
});

You can add the option isSearchable: true to your collection to display the search bar. Note that you will have to implement the search yourself by including it into your own get logic.

Implementing the GET (all records)

At this time, there’s no Smart Collection Implementation because no route in your admin backend handles the API call yet.

SQL

In the file routes/legal_docs.js, we’ve created a new route to implement the API behind the Smart Collection.

The logic here is to list all the files uploaded on a specific S3 Bucket. We use a custom service services/s3-helper.js for this example. The implementation code of this service is available on Github.

Finally, the last step is to serialize the response data in the expected format which is simply a standard JSON API document. We use the very simple JSON API Serializer library for this task.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

function reconcileData(file) {
  return models.documents
    .findOne({ where: { file_id: file.id } })
    .then((doc) => {
      file.is_verified = doc ? doc.is_verified : false;
      return file;
    });
}

router.get('/legal_docs', (req, res, next) => {
  return new S3Helper()
    .files('livedemo/legal')
    .then((files) => P.mapSeries(files, (file) => reconcileData(file)))
    .then((files) => Serializer.serialize(files))
    .then((files) => res.send(files))
    .catch((err) => next(err));
});

module.exports = router;

/serializers/legal_docs.js

const JSONAPISerializer = require('jsonapi-serializer').Serializer;

module.exports = new JSONAPISerializer('legal_docs', {
  attributes: ['url', 'last_modified', 'size', 'is_verified'],
  keyForAttribute: 'underscore_case',
});

Mongodb

In the file routes/legal_docs.js, we’ve created a new route to implement the API behind the Smart Collection.

The logic here is to list all the files uploaded on a specific S3 Bucket. We use a custom service services/s3-helper.js for this example. The implementation code of this service is available on Github.

Finally, the last step is to serialize the response data in the expected format which is simply a standard JSON API document. We use the very simple JSON API Serializer library for this task.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

function reconcileData(file) {
  return models.documents
    .findOne({ where: { file_id: file.id } })
    .then((doc) => {
      file.is_verified = doc ? doc.is_verified : false;
      return file;
    });
}

router.get('/legal_docs', (req, res, next) => {
  return new S3Helper()
    .files('livedemo/legal')
    .then((files) => P.mapSeries(files, (file) => reconcileData(file)))
    .then((files) => Serializer.serialize(files))
    .then((files) => res.send(files))
    .catch((err) => next(err));
});

module.exports = router;

/serializers/legal_docs.js

const JSONAPISerializer = require('jsonapi-serializer').Serializer;

module.exports = new JSONAPISerializer('legal_docs', {
  attributes: ['url', 'last_modified', 'size', 'is_verified'],
  keyForAttribute: 'underscore_case',
});

Implementing the GET (specific record)

SQL

To access the details view of a Smart Collection record, you have to catch the GET API call on a specific record. One more time, we use a custom service services/s3-helper.js that encapsulates the S3 business logic for this example.

The implementation of the reconcileData() and Serializer.serialize() functions are already described in the Implementing the GET (all records) section.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.get('/legal_docs/:doc_id', (req, res, next) => {
  return new S3Helper()
    .file(`livedemo/legal/${req.params.doc_id}`)
    .then((file) => reconcileData(file))
    .then((file) => Serializer.serialize(file))
    .then((file) => res.send(file))
    .catch((err) => next(err));
});

module.exports = router;

Mongodb

To access the details view of a Smart Collection record, you have to catch the GET API call on a specific record. One more time, we use a custom service services/s3-helper.js that encapsulates the S3 business logic for this example.

The implementation of the reconcileData() and Serializer.serialize() functions are already described in the Implementing the GET (all records) section.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.get('/legal_docs/:doc_id', (req, res, next) => {
  return new S3Helper()
    .file(`livedemo/legal/${req.params.doc_id}`)
    .then((file) => reconcileData(file))
    .then((file) => Serializer.serialize(file))
    .then((file) => res.send(file))
    .catch((err) => next(err));
});

module.exports = router;

Implementing the PUT

To handle the update of a record we have to catch the PUT API call. In our example, all S3-related fields are set as read-only and only is_verified can be updated.

SQL

The implementation of the reconcileData() and Serializer.serialize() functions are already explained in the Implementing the GET (all records) section.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.put('/legal_docs/:doc_id', (req, res, next) => {
  return models.documents
    .findOne({ where: { file_id: req.params.doc_id } })
    .then((doc) => {
      doc.is_verified = req.body.data.attributes.is_verified;
      return doc.save();
    })
    .then(() => new S3Helper().file(`livedemo/legal/${req.params.doc_id}`))
    .then((file) => reconcileData(file))
    .then((file) => Serializer.serialize(file))
    .then((file) => res.send(file))
    .catch((err) => next(err));
});

module.exports = router;

Mongodb

The implementation of the reconcileData() and Serializer.serialize() functions are already explained in the Implementing the GET (all records) section.

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.put('/legal_docs/:doc_id', (req, res, next) => {
  return models.documents
    .findOne({ where: { file_id: req.params.doc_id } })
    .then((doc) => {
      doc.is_verified = req.body.data.attributes.is_verified;
      return doc.save();
    })
    .then(() => new S3Helper().file(`livedemo/legal/${req.params.doc_id}`))
    .then((file) => reconcileData(file))
    .then((file) => Serializer.serialize(file))
    .then((file) => res.send(file))
    .catch((err) => next(err));
});

module.exports = router;

Implementing the DELETE

Now we are able to see all the legal documents on Forest Admin, it’s time to implement the DELETE HTTP method in order to remove the documents on S3 when the admin user needs it.

SQL

/routes/legal_docs.js

const express = require('express');
const router = express.Router();
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.delete('/legal_docs/:doc_id', (req, res, next) => {
  return new S3Helper()
    .deleteFile(`livedemo/legal/${req.params.doc_id}`)
    .then(() => res.status(204).send())
    .catch((err) => next(err));
});

module.exports = router;

Mongodb

/routes/legal_docs.js

const express = require('express');
const router = express.Router();
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.delete('/legal_docs/:doc_id', (req, res, next) => {
  return new S3Helper()
    .deleteFile(`livedemo/legal/${req.params.doc_id}`)
    .then(() => res.status(204).send())
    .catch((err) => next(err));
});

module.exports = router;

Implementing the POST

On our Live Demo example, creating a record directly from this Smart Collection does not make any sense because the admin user will upload the legal docs in the company details view. For the documentation purpose, we catch the call and returns an appropriate error message to the admin user.

SQL

/routes/legal_docs.js

...

router.post('/legal_docs', permissionMiddlewareCreator.create(), (request, response) => {
  response.status(400).send('You cannot create legal documents from here. Please, upload them directly in the details view of a Company');
});

...

module.exports = router;

Mongodb

/routes/legal_docs.js

const P = require('bluebird');
const express = require('express');
const router = express.Router();
const models = require('../models');
const S3Helper = require('../services/s3-helper');
const Serializer = require('../serializers/legal_docs');

// ...

router.post('/legal_docs', (req, res, next) => {
  res
    .status(400)
    .send(
      'You cannot create legal documents from here. Please, upload them directly in the details view of a Company'
    );
});

module.exports = router;