Toggle menu

Data Storage - What and Where?

The services running on your digital platform, whether via products we've developed like Case Management or Bookings, or forms and processes you have designed yourself, have the potential to collect a lot of personal data from users. This series of articles describes the platform services that store and manage data, the things you need to be aware of, and the different tools available to retrieve and delete data.

General Principles

The products and services that run on the digital platform are very flexible and infinitely customisable. Even our out-of-the-box products will often let you plug-in your own forms for users to complete. That makes it impossible for this documentation to list all of the personal data a product collects - only the person designing the service can know that.

When you design and deliver a service using the platform, you need to know what data you are collecting and storing. That's not as daunting as it sounds. Third-party integrations aside, the main way personal data from users will get into the platform is via form submissions. If you make your website's forms part of your personal data inventory (opens new window), tracking the data you are collecting will become second nature.

As always, the information you collect from a user should always be adequate, relevant, and limited to the purpose for which it is needed to provide the service.


All of the data used, created, stored and returned from the platform is, ultimately, held in the iCM database. The database can only be accessed using our APIs, generally using the worker services available in iCM's API Server.

Some data may also be indexed by the iCM search. We consider this data store transient, because as data is deleted or updated in other areas of iCM (for example, workflow processes) the search collections are automatically updated by iCM too.


Form Actions and Storage

StoreDescriptionTypical useRetentionExport
HistoryHistory is designed for long term storage. Histories are unique logs of timestamped events. An event can store any data, as long as it's valid JSON. Once written a history is immutable. A history can be "sealed" preventing further events being written to itHistories are generally created when a workflow process starts. As the workflow process progresses, new events are written to the corresponding history. When the process ends, the history is sealed. The history provides a record of what happened during the process.

A single "sealed" history can also be written by a stand-alone form submission using the History Write form field
History retention policies are set up and maintained by the Data Retention ManagerThe Data Exporter can export history data as CSV or JSON
Form Data as iCM ObjectsUnlike histories, objects have a defined structure (ie named fields/properties) to store data. Stored values can be updated and also searched/managed within iCMObjects can be created and updated directly using the iCM API, but are more commonly created by saved form data using the Database Save form fieldThe Data Retention Manager does not have policies for iCM Objects. You can search for and delete objects from within the iCM interfaceForm data can be searched for and exported as a CSV from within the iCM interface
User ProfilesEvery registered site user has a profile. Profiles are managed using forms and save their data as iCM ObjectsProfiles have a set of standard fields, including name, email, address and phone numbers. Only the email address is mandatoryUsers can be deleted from within iCM, which will also delete their profileUsers and their profiles can be exported as a CSV file from within iCM
WorkflowWorkflow data should be thought of as transient. Data is stored in a process instance using process variables, which can be updated in active processes as the workflow progressesThe workflow engine is designed to coordinate services involving tasks, decisions, timers and automated processes. Process instances will include data from form submissions that interact with the processThe Data Retention Manager can optionally delete process data at the same time as the accompanying history record. By default, completed process instances over a year old are deleted by an iCM scheduled taskWhile workflow data can be queried by the API, there is no export function
FilestoreThe Filestore is an internal component used by other services to securely encrypt and store filesFiles uploaded from form submissions, or as part of workflows, are stored in the Filestore. Files remain in the store until no other services (forms, workflow, history, objects) have need of themFiles are deleted by whatever uploaded them when they are no longer neededFiles should only be accessed via the published interfaces/other services running on the platform


User Profiles

When a user registers with your site, as a minimum you'll store their email address and preferred name as a website user.

History Service

The API Server's History worker stores a log of events, called a history, with a unique id called a key. Anything capable of communicating with the API Server can potentially access the History worker and record data.

Workflow Processes

Workflow processes store data in process variables, which are updated throughout the lifetime of a process instance.

Form Data and iCM Objects

iCM Objects are defined data structures that can be written to by form submissions or the API.


The filestore provides secure, encrypted storage for files uploaded to the platform, primarily via form submissions.

Share this page

Facebook icon Twitter icon email icon


print icon