# Replication strategy

{% hint style="success" %}
This is the official documentation of the `@forestadmin/agent` Node.js agent.
{% endhint %}

![Minimal example in action](/files/x6gheIToGhnxYrISMgLD)

The idea behind data sources using the "replication" strategy is to maintain a copy of the data from the target API in an internal cache that is under the control of your agent.

| Pros             | Why?                                                                                              |
| ---------------- | ------------------------------------------------------------------------------------------------- |
| Easy             | No need to implement any query translation logic.                                                 |
| Performant       | No need to make network calls to the target API synchronously.                                    |
| Feature-complete | Charts, filtering, search is provided out of the box.                                             |
| Flexible         | You can implement any logic you want to fetch the data from the target API.                       |
| Robust           | Any bad state can be recovered from by reconstructing the replica from scratch in the background. |

## Minimal example

This example shows how to create a custom data source using the "replication" strategy on [{JSON} Placeholder](https://jsonplaceholder.typicode.com/), a free API that returns read-only fake data.

In this trivial example, records will be fetched from the target API when the agent first starts, never to be updated again.

### Implementation

The most stripped-down implementation of a data source using the "replication" strategy is simply providing a function that returns a list of entries to be imported.

```javascript
const { createReplicaDataSource } = require('@forestadmin/datasource-replica');
const axios = require('axios');

// Create the data source
const myCustomDataSource = createReplicaDataSource({
  pullDumpHandler: async () => {
    const url = 'https://jsonplaceholder.typicode.com';
    const collections = ['posts', 'comments', 'albums', 'photos', 'users'];
    const entries = [];

    for (const collection of collections) {
      const response = await axios.get(`${url}/${collection}`);
      entries.push(...response.data.map(record => ({ collection, record })));
    }

    return { more: false, entries };
  },
});

// Register the data source with the agent
agent.addDatasource(myCustomDataSource);
```

### Limitations

This minimal example is a good starting point to understand the selling points of the replication strategy, however, strong limitations remain and you may need to implement more logic to tackle them.

| Limitation                                                             | Mitigation                                                                                                                             |
| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| Needs to dump all records at each startup                              | Use a [persistent cache](/developer-guide-agents-nodejs/data-sources/custom/replication/persistent-cache.md)                           |
| Empty collections cannot be imported and foreign keys are not detected | Provide a [schema](/developer-guide-agents-nodejs/data-sources/custom/replication/schema.md)                                           |
| The data is never updated                                              | Implement [updates](/developer-guide-agents-nodejs/data-sources/custom/replication/updates.md)                                         |
| The data is readonly                                                   | Implement [write handlers](/developer-guide-agents-nodejs/data-sources/custom/replication/write.md)                                    |
| Records from API may contain nested fields and arrays                  | Use the [record flattener](/developer-guide-agents-nodejs/data-sources/custom/replication/schema.md#handling-arrays-and-nested-fields) |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.forestadmin.com/developer-guide-agents-nodejs/data-sources/custom/replication.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.