File system templates store

The default template store implementation persist entities into file system. It is designed to conveniently generate human readable directory structure with templates and other entities definitions. Such structure is very easy to edit with your favorite text editor, deploy to the production and also version with git or other source control. The file system implementation finally supports also cloud based "storage" like AWS S3 or Azure Storage.

Configuration

The default configuration is present in the jsreport.config.json file pre-created during the installation.

"store": { "provider": "fs" }

Additionally you can also specify which directory should be used for persisting entities and if the changes should be automatically synchronized with the UI.

"store": {
  "provider": "fs"  
},
"extensions": {
  "fs-store": {
    "dataDirectory": "data",
    "syncModifications": true
  }
}

Directory structure

The directory structure generated by fs store is designed to be easy to read. The most of the entities like templates resides in the specific folder which includes config.json file with meta attributes like template recipe or engine. The properties which includes the whole document like template helpers are extracted into dedicated file. This means that you can find for example the template's helpers in the helpers.js.

data \ myfolder

    • invoice
        • content.html
        • helpers.js
        • config.json

Every folder inside the data directory that isn't representing an entity is represented as folder also in the jsreport studio. And every file that isn't inside an entity folder is representing an asset that is also visible in the jsreport studio. Such assets are hover not having extra properties so you can edit only its content.

Deployment

The deployment of the stored templates to the production server can be done just by copy pasting the data directory. This is also very convenient way to backup the production templates.

Editing templates in custom editor

The fs store monitors changes in the underlying files and notifies the studio UI to reload and re-render the active template through sockets. This behavior makes it very easy to edit the templates' code using your editor and immediately see the rendered output in the jsreport studio.

editing

Note this behavior is enabled only in the development environment by default, although you can override this using extensions.fs-store.syncModifications = true config.

You can configure how file monitoring should be done by editing sync.usePolling config. when true it uses polling strategy (checks file stats every x time), when false it uses native os watching mechanism. defaults to true. if you detect some problem with file watching in your installation you can try to set sync.usePolling to false and check if it makes the problem disappear.

"store": {
  "provider": "fs"  
},
"extensions": {
  "fs-store": {
    "dataDirectory": "data",
    "syncModifications": true,
    "sync": {
      "usePolling": true
    }
  }
}

Source control

The folder structure created by the fs store is designed to be human readable and easily versioned using source controls like git. The integration is just about including the app/data folder to the source control repository.

Consistency

The fs store uses transnational approach for writing entities to assure that the underlying data keeps consistent even if the process crashes in the middle. Technically it always create the new folder with changed entity and rename it to the final name after everything succeeds.

Running in cluster

The fs store is designed to support running multiple jsreport instances over the same directory. It is assured that parallel request doesn't break the data consistency using lock files. The changes made from another process are automatically monitored and the memory representation reloaded. This also allows to map a network drive and single data folder to multiple jsreport instances.

Cloud storage

The fs store by default writes entities to the file system but it also includes persistence abstraction which can be implemented by another extension. Such extension can then provide persistence implementation for another media like AWS S3 or Azure Blob Storage.

jsreport using such extension loads all entities only during the startup and then use in memory representation for queries. This makes typical operations like loading and rendering templates very fast. However it doesn't work in the environment with multiple instances writing to the single storage. Fortunately fs store provides another abstraction used for synchronizing changes between servers. This is technically implemented using cloud native service bus.

AWS S3

You need to first create an IAM user with permissions to S3/SNS/SQS and copy the access key with the secret access key. Then create a bucket and copy its name.

To configure jsreport persisting templates into AWS S3 you need to install jsreport-fs-store-aws-s3-persistence extension first.

npm install jsreport-fs-store-aws-s3-persistence

The next step is to configure the fs store persistence using the following options. The jsreport instance should then load and persist templates to specified bucket.

"store": {
  "provider": "fs"
},
"extensions": {
  "fs-store": {
    "persistence": {
      "provider": "aws-s3"
    }
  },
  "fs-store-aws-s3-persistence": {
    "accessKeyId": "...",
    "secretAccessKey": "...",
    "bucket": "....",
    "lock": {
      "queueName": "jsreport-lock.fifo"     
    }
  }
}

To configure jsreport persisting templates in S3 to run in the multi server environment you need to install jsreport-fs-store-aws-sns-sync extension.

npm install jsreport-fs-store-aws-sns-sync

The second step is to configure the fs store synchronization using the following options.

"store": {
  "provider": "fs"
},
"extensions": {
  "fs-store": {
    "persistence": {...},
    "sync": {
      "provider": "aws-sns"
    }
  },
  "fs-store-aws-sns-sync": {
    "accessKeyId": "...",
    "secretAccessKey": "..."   
  }
}

This assures all servers gets a message and reflect it to the memory representation every time there is a change written to the S3. These messages are delivered to the jsreport instances using the native AWS SNS service.

You can find further information in the following repositories.

jsreport-fs-store-aws-s3-persistence
jsreport-fs-store-aws-sns-sync

Azure Storage

You need to create an azure storage account and copy the account name and the access key first.

To configure jsreport persisting templates into Azure Blob Storage you need to install jsreport-fs-store-azure-storage-persistence extension.

npm install jsreport-fs-store-azure-storage-persistence

The second step is to configure the fs store persistence using the following options. The jsreport instance should then load and persist templates to Azure Blob Storage jsreport container.

"store": {
  "provider": "fs"
},
"extensions": {
  "fs-store": {
    "persistence": {
      "provider": "azure-storage"
    }
  },
  "fs-store-azure-storage-persistence": {
    "accountName": "...",
    "accountKey": "..."  
  }
}

To configure jsreport persisting templates into Azure Blob Storage and run in the multi server environment you need to first install jsreport-fs-store-azure-sb-sync extension.

npm install jsreport-fs-store-azure-sb-sync

Then create an azure service bus and copy the connection string from the shared access policies. The next step is to configure the fs store synchronization using the following options.

"store": {
  "provider": "fs"
},
"extensions": {
  "fs-store": {
    "sync": {
      "provider": "azure-sb"
    }
  },
  "fs-store-azure-sb-sync": {
    "connectionString": "..."    
  }
}

This assures all servers gets a message and reflect it to the memory representation every time there is a change written by one of the instance to the Azure Blob Storage. These messages are delivered to the jsreport instances using the native Azure Service Bus.

You can find further information in the following repositories.

jsreport-fs-store-azure-storage-persistence jsreport-fs-store-azure-sb-sync