The services running on your digital platform, whether via products we've developed like Case Management or Bookings, or forms and processes you have designed yourself, have the potential to collect a lot of personal data from users. This series of articles describes the platform services that store and manage data, the things you need to be aware of, and the different tools available to retrieve and delete data.
General Principles
The products and services that run on the digital platform are very flexible and infinitely customisable. Even our out-of-the-box products will often let you plug-in your own forms for users to complete. That makes it impossible for this documentation to list all of the personal data a product collects - only the person designing the service can know that.
When you design and deliver a service using the platform, you need to know what data you are collecting and storing. That's not as daunting as it sounds. Third-party integrations aside, the main way personal data from users will get into the platform is via form submissions. If you make your website's forms part of your personal data inventory (opens new window), tracking the data you are collecting will become second nature.
As always, the information you collect from a user should always be adequate, relevant, and limited to the purpose for which it is needed to provide the service.
Where?
All of the data used, created, stored and returned from the platform is, ultimately, held in the iCM database. The database can only be accessed using our APIs, generally using the worker services available in iCM's API Server.
Some data may also be indexed by the iCM search. We consider this data store transient, because as data is deleted or updated in other areas of iCM (for example, workflow processes) the search collections are automatically updated by iCM too.
Summary
| Store | Description | Typical use | Retention | Export |
|---|---|---|---|---|
| History | History is designed for long term storage. Histories are unique logs of timestamped events. An event can store any data, as long as it's valid JSON. Once written a history is immutable. A history can be "sealed" preventing further events being written to it | Histories are generally created when a workflow process starts. As the workflow process progresses, new events are written to the corresponding history. When the process ends, the history is sealed. The history provides a record of what happened during the process. A single "sealed" history can also be written by a stand-alone form submission using the History Write form field | History retention policies are set up and maintained by the Data Retention Manager | The Data Exporter can export history data as CSV or JSON |
| Form Data as iCM Objects | Unlike histories, objects have a defined structure (ie named fields/properties) to store data. Stored values can be updated and also searched/managed within iCM | Objects can be created and updated directly using the iCM API, but are more commonly created by saved form data using the Database Save form field | The Data Retention Manager does not have policies for iCM Objects. You can search for and delete objects from within the iCM interface | Form data can be searched for and exported as a CSV from within the iCM interface |
| User Profiles | Every registered site user has a profile. Profiles are managed using forms and save their data as iCM Objects | Profiles have a set of standard fields, including name, email, address and phone numbers. Only the email address is mandatory | Users can be deleted from within iCM, which will also delete their profile | Users and their profiles can be exported as a CSV file from within iCM |
| Workflow | Workflow data should be thought of as transient. Data is stored in a process instance using process variables, which can be updated in active processes as the workflow progresses | The workflow engine is designed to coordinate services involving tasks, decisions, timers and automated processes. Process instances will include data from form submissions that interact with the process | The Data Retention Manager can optionally delete process data at the same time as the accompanying history record. By default, completed process instances over a year old are deleted by an iCM scheduled task | While workflow data can be queried by the API, there is no export function |
| Filestore | The Filestore is an internal component used by other services to securely encrypt and store files | Files uploaded from form submissions, or as part of workflows, are stored in the Filestore. Files remain in the store until no other services (forms, workflow, history, objects) have need of them | Files are deleted by whatever uploaded them when they are no longer needed | Files should only be accessed via the published interfaces/other services running on the platform |
User Profiles
History Service
Workflow Processes
Form Data and iCM Objects
Filestore
Share this page
