Scheduled rebuilds
This is the official documentation of the @forestadmin/agent
Node.js agent.
The simplest way to update the data is to fetch all the records from the target API at regular intervals.
This approach is trivial to implement and should work with any API, as the only requirement is to be able to fetch all the records from the target API.
Its main drawback is that it is not very efficient: if you have a large number of records, you will need to fetch all of them at each interval, even if only a few of them changed.
Choosing when to rebuild your replica
This can be achieved with two options:
pullDumpOnRestart
: if set totrue
, the data will be fetched each time the agent starts. This is always true if you are using the default in-memory cache.pullDumpOnSchedule
: if set to a cron-like schedule, the data will be fetched at the given interval
const myCustomDataSource = createReplicaDataSource({
// Update the data each time the agent starts.
pullDumpOnRestart: true,
// Update the data each day at midnight and 6.30pm.
pullDumpOnSchedule: ['0 0 0 * * *', '0 30 18 * * *'],
// Handler to fetch all records from the target API.
pullDumpHandler: async () => {
// Implement handler here
},
});
Programming your handler
This handler is the simplest one to implement: it is simply expected to return a list of entries to be imported, each entry containing the collection name and the record to import.
As the handler may return a large amount of data, you are allowed to paginate the results.
const myCustomDataSource = createReplicaDataSource({
// Update the data each time the agent starts.
pullDumpOnRestart: true,
// Update the data each day at midnight and 6.30pm.
pullDumpOnSchedule: ['0 0 0 * * *', '0 30 18 * * *'],
// Handler to fetch all records from the target API.
pullDumpHandler: async () => {
const url = 'https://jsonplaceholder.typicode.com';
const collections = ['posts', 'comments', 'albums', 'photos', 'users'];
const entries = [];
for (const collection of collections) {
const response = await axios.get(`${url}/${collection}`);
entries.push(...response.data.map(record => ({ collection, record })));
}
return { more: false, entries };
},
});
Last updated
Was this helpful?