How to Easily Edit Data on Devices Using Export and Import Features
In this tutorial, we'll guide you through the process of correcting data for Immutable devices using TagoIO's export and import features. Since Immutable devices don't allow editing individual variables directly, this method provides an effective workaround to rectify data issues.
For example, let's say you need to edit a variable named 'group', which is used to organize variables in widgets. Due to a problem with my payload parser, data was added to my Immutable bucket with incorrect grouping. This caused my Dynamic List widget to display the variables in separate rows, as if they were distinct from one another.
Here's how to fix it using export and import features.
Step 1: Export the Data in CSV Format
1. Navigate to the Device module and select the Immutable device that contains the data you wish to correct.
This method is also applicable to Mutable devices, as the underlying principles remain consistent. The key difference is that with Mutable buckets, you have greater flexibility to remove individual data points rather than entire chunks.
This method is also applicable to Mutable devices, as the underlying principles remain consistent. The key difference is that with Mutable buckets, you have greater flexibility to remove individual data points rather than entire chunks.
2. Click on the 'Chunks' tab. Select the chunk of data you'd like to fix, and click on 'Export chunks'. This is the data you need to manually correct.
3. The chunk of data will be exported to your Files at the address you've specified. It may take a few minutes. Once the .csv file is in your Files, you can download and edit it.
This will consume your Data Output service.
Step 2: Promptly Delete Current Device Data After Exporting
To prevent duplicate entries when re-importing corrected data, it's important to delete the export data from your device.
We recommend doing this promptly, especially if your device is actively collecting new data. Since your data is securely stored in your files, you can safely proceed with the deletion. Delaying this step until after making manual edits could result in the loss of any new sensor data that might be transmitted while you're working with the data in your environment.
We recommend doing this promptly, especially if your device is actively collecting new data. Since your data is securely stored in your files, you can safely proceed with the deletion. Delaying this step until after making manual edits could result in the loss of any new sensor data that might be transmitted while you're working with the data in your environment.
1. Return to the device's 'Chunks' tab.
2. Select the chunk of data you previously exported, and click on 'Delete Chunks'.
3. Confirm the action when prompted. Ensure it is the correct chunk, as this action is irreversible.
Step 3: Apply the Corrections
1. Open the exported CSV file in a spreadsheet application or any data manipulation tool. You can use Excel, Google Sheets, or even programming languages like Python or SQL for more complex edits.
Remember to maintain the required CSV format:
- Keep the headers intact, especially the mandatory 'variable' field.
- Ensure values are separated by commas.
- Save the corrected file in CSV format.
- The file size should not exceed 200 MB.
Step 4: Import the Corrected Data
Now, let's repopulate your device with the corrected data by uploading the CSV file.
1. Go to the 'Data' tab of your device. Click the 'More' button in the top right corner, then select 'Restore from Files'.
2. Click 'Upload File' and select your corrected CSV file.
3. Once uploaded, choose the corrected CSV file you just uploaded.
4. Click on 'Confirm'.
And that's it! Now that the data is corrected with the proper grouping, my data is displayed correctly.
Important Notes on Importing Data
- Ensure your corrected data falls within the chunk period and retention configuration of your Immutable device. Data outside this period will cause an import error.
- The device payload parser doesn't execute during data import, so ensure your data is properly formatted before importing.
- The CSV file should not exceed 200 MB and should contain a maximum of 1 million rows.
- You must have at least 10,000 Data Storage registers available to perform the import, even if you're importing less data.
When to use this method?
This method is particularly useful for fixing Immutable device data in various scenarios, such as:
- Populating old data from your device that wasn't captured with TagoIO.
- Correcting variables that were sent incorrectly.
- Resolving parsing issues where your payload parser didn't process the data as intended.
- And more!
If you have any questions, please let us know by replying to this topic.
If you have any questions, please let us know by replying to this topic.