Device data management
The device page provides tools for visualizing, filtering, editing, importing and exporting variables, facilitating the management of data received by your device. You can access these tools by navigating to the Device module, selecting a device, and clicking on the 'Data' tab.
Amount of data records
The amount of data stored in each device is displayed at the top of the page. When the number of data records exceeds 1,000 registers, TagoIO will start to display a truncated value. This represents an estimated number of data registers, not an exact one.
By accessing the Device module, you will find a list of your devices and the amount of data each contains. These numbers are also truncated when greater than 1,000 registers. The values might include letters representing the number of decimal places, where "K" stands for thousand, "M" for million, "B" for billion, and "T" for trillion.
Data stored on your devices consumes your Data Storage service. You can visualize how much of the limit is left or upgrade it by going to the Billing Page in your Admin.Managing variable data
The 'Data' tab on your device page lists all stored variables, including their names, values, group, metadata, and time. Here, you can inspect each variable individually, as well as import and export them. For Mutable devices, you can even edit or delete them individually.
Filtering
Filter options within the 'Data' tab enable you to refine your variable search based on specific criteria.
The table visualization can be customized by adjusting column visibility or changing the time format, this option is accessible through the cog icon in the table's lower-left corner.
Editing and deleting variables individually
For mutable devices, individual variables can be edited or deleted directly from this tab. Simply hover over the desired information and click the pencil icon to make edits.
To delete variables, first select the ones you want to remove. Once selected, click on 'Delete selected' to proceed with the deletion. Once done, this action is irreversible.
To delete variables, first select the ones you want to remove. Once selected, click on 'Delete selected' to proceed with the deletion. Once done, this action is irreversible.
For immutable devices, it is not possible to edit data individually; data can only be deleted through the data chunks. Read more about it here: Chunk Management.
Importing
To import data in bulk into your device, navigate to the 'Data' tab and click the 'More' button in the top right corner. Then, select 'Restore from Files'. Before importing, ensure that your data is uploaded to your Files. If you find that you don't have sufficient storage space, you can easily increase your limit by visiting the Billing page in your Admin panel.
The supported file format is CSV.
The device payload parser does not execute during data import. Therefore, ensure that the data is properly formatted before importing.
CSV format requirements
To ensure a successful import, your CSV file must adhere to the following requirements:
- Include headers. The 'variable' field is mandatory, while others are optional.
- Values should be separated by a comma. Semicolons are not supported.
- The file size should not exceed 200 MB.
- The file should contain a maximum of 1 million rows.
- Data integrity and structure should align with TagoIO's data format. For more information, refer to the Sending Data documentation.
The following headers are available for import. Any additional headers beyond those listed will be ignored:
| variable (mandatory) |
| value |
| group |
| unit |
| location |
| metadata |
| time |
Service limits for importing
When importing data, the Data Input limit is not affected. However, since you are adding new data to your devices, the amount of new data will count against your Data Storage service. You must have at least 10,000 Data Storage registers available to perform the import, even if you are importing less data than that.
Additionally, before uploading the data to your device, you must first upload it to your Files. Ensure you have enough space to upload your CSV file.
If needed, you can increase the data storage or file limits by accessing the Billing page in your Admin.
Differences between importing data to Mutable vs. Immutable Devices
TagoIO supports two types of devices: Mutable and Immutable. Each has specific considerations when importing data.
- Immutable Devices: If your CSV includes data for the 'time' field, the time must fall within your chunk period and retention configuration. If you attempt to import data outside the retention period, you will encounter an error. To resolve this, ensure the date and time information fits within your retention period.
- Mutable Devices: These devices can store up to 50,000 records. Your import will fail if this limit is exceeded. You can check the number of data records in your bucket in the Device module.
Exporting
The ‘Export’ button, located in the top right corner of the ‘Data’ tab, allows for variable data export in various formats. Further details on exporting data for specific device types can be found under: Data Export.
Backing up your data
To back up your device data, click the 'More' button located in the top right corner, then select 'Backup to Files' under the 'Data' tab. This will export all your bucket data to your Files, creating a secure copy of your sensor data. This feature is particularly useful for safeguarding your data or capturing snapshots at specific points in time.
The tool automatically backs up all your device data. If you need to apply filters before backing up, use the Data Export function to refine your data, and then save it locally or in your Files. For Immutable devices, clicking the 'Backup to Files' button will redirect you to the 'Chunk Management' tab, where you can select specific chunks of data to back up to your Files.
File Address Template
During the backup process, you can customize the file address before exporting. As a result, each backup file will have its own unique file address. In the Advanced Settings, you can use key variables to modify the path name that will be created within your Files. The variables available for customizing your file address are listed below:
$DEVICE$ - the device ID
$CHUNK$ - the chunk ID
$FROM$ - the chunk start date
$TO$ - the chunk end date
For example, the following file address template:
- device/$DEVICE$/$FROM$_$TO$
Would result in a file address such as:
- /device/6297aa7691e70a00654f9816/2022-04-01_2022-06-30.csv
Date variables such as $FROM$ and $TO$ will be in the ISO 8601 format (YYYY-MM-DD)
When exporting a chunk to a location where a file already exists, all the data will be replaced with the new one.
Emptying your Device Data
While it's feasible to remove variables one by one, there's a more efficient method to wipe all data simultaneously.
Navigate to the Device module and choose the device you wish to clear. Then, click the 'More' button located in the top right corner, then select 'Empty Device Data' under the 'Data' tab.
Related Articles
Sending Data to Device
TagoIO supports two-way communication with all your devices so that it can send and receive data. There are some options for you to send data from your application to the device. Among the methods, you can use: Publish to MQTT Post data to a ...Data Export
At TagoIO, you have multiple options for exporting data. You can export device data directly from the Admin panel on your device's page, through the API, or directly from Widgets in either the Admin panel or the TagoRun portal. Exporting data will ...Chunk Management
The Chunk Management feature is a powerful tool that allows you to easily manage the data of your Immutable devices, either to create backups or delete information. By accessing the Device module and selecting a Device Data Optimized (Immutable) ...Device Emulator
The device emulator is a powerful tool for developers to send data as if it was sent by a real device. The emulator is intuitive and requires just a few clicks to validate and send your data package. To access the device emulator, go to the Device ...Entities
The Entity feature is an excellent tool for storing data in a tabular format. Unlike the Devices feature, which is optimized for storing sensor data with high-frequency write operations, the Entity feature offers a versatile solution for managing a ...