Chunk Management [beta]

Chunk Management [beta]

The Chunk Management feature is a powerful tool that allows you to easily manage your data, either to create backups or delete information from your bucket.
Chunk Management is only available for Optimized Device Data (Immutable) 
By accessing your Bucket module and selecting a Device Data Optimized (Immutable) bucket type from your list, you will be able to find the Chunk Management section where there's a graph that shows your data chunks based on the period you chose when creating your device.

Export chunks to TagoIO Files

By selecting one or multiple chunks, you will be able to export them to your TagoIO Files in CSV format to create a backup of your data. In this process, the chunks are added to a queue and soon become available on the file address.


File Address Template

During the exportation process, you will be able to customize the file address before exporting your data to TagoIO Files, and as a result, each chunk file will have its own file address. In the Advanced Settings, you can use key variables to change the path name that will be created inside your TagoIO Files. The variables you can use to customize your file address are listed below:

  • $DEVICE$ - the device ID

  • $CHUNK$ - the chunk ID 

  • $FROM$ - the chunk start date

  •  $TO$ - the chunk end date


For example, the following file address template:
  1. device/$DEVICE$/$FROM$_$TO$

Would result in a file address such as:
  1. /device/6297aa7691e70a00654f9816/2022-04-01_2022-06-30.csv
Date variables such as $FROM$ and $TO$ will be in the ISO 8601 format (YYYY-MM-DD)
When exporting a chunk to a location where a file already exists, all the data will be replaced with the new one.


Include Header

Activating the Include Header option will enable headers on CSV-generated files. Headers will describe the variable's data in each column, just like a column's name, but you can disable them in the Advanced Settings to keep your files filled only with rows of data.

How exporting is billed

Notice that exporting data consumes the Data Output service. The number of transactions billed is rounded to the next multiple of 10,000. For example:
  1. Exporting 700 registers will consume 10,000 data output transactions.
  2. Exporting 12,000 registers will consume 20,000 data output transactions.
  3. Exporting 32,000 registers will consume 40,000 data output transactions.
  4. Exporting 103,000 registers will consume 110,000 data output transactions.


Delete chunks

You can delete chunks to optimize your data bucket storage. This is an irreversible process, and once done there is no going back, so make sure to be certain before removing anything. You can check the chunk range information by hovering your mouse over the graph before deleting the data.
You can keep track of all interactions made on this feature through the Audit Log.


Automation

You can also access the chunk management functions using the SDK, Analysis, and Actions to automate processes to either backup or delete your data.
    • Related Articles

    • Team Management - Sharing your profile

      You can grant access to others to access any of your profiles by using the Team feature. By adding others to your team, they will have access to all the applications that run on that profile.  Only the owner of the Profile is able to invite members. ...
    • Data Retention Feature

      The Data Retention feature automatically removes old data from the bucket after the period you define here. The goal is to help customers to avoid unnecessary costs with data that don’t need to be kept for a long period of time. ​ The Data Retention ...
    • Resetting My Password

      On the signup page, click on the Forgot your password? link and enter your email, and we'll send you instructions to reset your password. Want to change your password? To change your existing password, go to My Account, select the tab 'Account ...
    • End-Users service

      Each end-user that is added in the User Management list is counted as an end-user.  End-users, also called RUN-users, can be automatically added when they sign up in your RUN application. If any of your profile reaches the limit of end-users ...
    • Audit Log

      Use this feature to keep up with all activities of a profile owned by you. Each activity as creating a new Device, updating an Access Management policy, updating an Analysis, and more, will keep a log and will be shown in the Audit Log.  Filtering ...

    Webinar Announcement - Image Only

        Take a look at our updated roadmap here!
          What would you like to see on TagoIO?

            Future and past Webinars