Chunk Management

Chunk Management

The Chunk Management feature is a powerful tool that allows you to easily manage your data, either to create backups or delete information from your bucket.
Chunk Management is only available for Optimized Device Data (Immutable) 
By accessing your Bucket module and selecting a Device Data Optimized (Immutable) bucket type from your list, you will be able to find the Chunk Management section where there's a graph that shows your data chunks based on the period you chose when creating your device.

Export chunks to TagoIO Files

By selecting one or multiple chunks, you will be able to export them to your TagoIO Files in CSV format to create a backup of your data. In this process, the chunks are added to a queue and soon become available on the file address.


File Address Template

During the exportation process, you will be able to customize the file address before exporting your data to TagoIO Files, and as a result, each chunk file will have its own file address. In the Advanced Settings, you can use key variables to change the path name that will be created inside your TagoIO Files. The variables you can use to customize your file address are listed below:

  • $DEVICE$ - the device ID

  • $CHUNK$ - the chunk ID 

  • $FROM$ - the chunk start date

  •  $TO$ - the chunk end date


For example, the following file address template:
  1. device/$DEVICE$/$FROM$_$TO$

Would result in a file address such as:
  1. /device/6297aa7691e70a00654f9816/2022-04-01_2022-06-30.csv
Date variables such as $FROM$ and $TO$ will be in the ISO 8601 format (YYYY-MM-DD)
When exporting a chunk to a location where a file already exists, all the data will be replaced with the new one.


Include Header

Activating the Include Header option will enable headers on CSV-generated files. Headers will describe the variable's data in each column, just like a column's name, but you can disable them in the Advanced Settings to keep your files filled only with rows of data.

How exporting is billed

Notice that exporting data consumes the Data Output service. The number of transactions billed is rounded to the next multiple of 10,000. For example:
  1. Exporting 700 registers will consume 10,000 data output transactions.
  2. Exporting 12,000 registers will consume 20,000 data output transactions.
  3. Exporting 32,000 registers will consume 40,000 data output transactions.
  4. Exporting 103,000 registers will consume 110,000 data output transactions.


Delete chunks

You can delete chunks to optimize your data bucket storage. This is an irreversible process, and once done there is no going back, so make sure to be certain before removing anything. You can check the chunk range information by hovering your mouse over the graph before deleting the data.
You can keep track of all interactions made on this feature through the Audit Log.


Automation

You can also access the chunk management functions using the SDK, Analysis, and Actions to automate processes to either backup or delete your data.

    • Related Articles

    • Team Management - Sharing your profile

      You can grant access to other TagoIO accounts to any of your Profiles by using the Team feature. By adding others to your team, they will have access to all the applications that run on that profile.  The invitation can only be sent to emails that ...

    • Rate Limits (Hard limits)

      When executing requests to TagoIO, you will have a limit on the number of requests that can be made during a certain time period. These limits are applied in short intervals every minute, allowing us to provide a reliable and scalable API that our ...

    • Data Export

      The Export Data feature enables you to efficiently extract variables from your bucket. This function is accessible within the Admin panel, specifically in the Bucket Module . To use it, select a bucket and navigate to the ' Variables' tab. On this ...

    • End-Users service

      Each end-user that is added in the User Management list is counted as an end-user.  End-users, also called RUN-users, can be automatically added when they sign up in your RUN application. If any of your profile reaches the limit of end-users ...

    • Resetting My Password

      On the signup page, click on the Forgot your password? link and enter your email, and we'll send you instructions to reset your password. Want to change your password? To change your existing password, go to My Account, select the tab 'Account ...