Skip to content

Delete Parquet Data Source

A Parquet Data Source can be deleted from the Data Lake folder using the Delete option in the IFS Cloud Web Parquet Data Source / Parquet Data Source Details page.

A Data Source could be,

Only the Data Sources which are not connected to any Analysis Model or a Workload Job Definition can be deleted. (Also the Data Sources with Base configuration state cannot be removed as well)

Deleting a Parquet Data Source

1.Select the Parquet Data Source which requires to be deleted.

2.Use the Delete option from the Parquet Data Source / Parquet Data Source Details page.

3.Prompt is displayed with the below details.

3.1 If the Data Source is connected to one or more Analysis Models, Deleting is not allowed.

3.2 If the Data Source is connected to one or more Workload Job Definitions, Deleting is not allowed.

3.3 If the Data Source is not connected to any Analysis Model or a Workload Job Definition, Deleting is allowed.

Click OK to proceed with the Deletion.

During the process, the Delete option is disabled.

4.Mark for Deletion is set to Yes.

Scheduler checks Data Source(s) which are marked for deletion and the Data Source Refresh History status will be updated as, Job Queued For Deleting. This will add the job to the queue to delete the file from the Data Lake. Once the Data Pump picks up the job, the respective status will be updated as Deleting.

(status transition: Current Data Source Refresh History status > Job Queued For Deleting > Deleting).

Parquet Data Source status : Job Queued For Deleting

Parquet Data Source status : Deleting

5.Once the process is completed, the Parquet Data Source entry will be removed from the Parquet Data Source page, and the Data Source load history will be updated with the Deleted status in the database.

And the Parquet file / folder which holds the parquet file will be removed from all Data Lakes where the Data Source was loaded. This will be indicated in the Refresh History Destination field when the Refresh History status is set to Job Queued for Deleting and Deleting. It ensures that Data Source is removed from all the destinations, as the Destination value could be switched during each instance of Loading.

Note

When the Parquet Data Source Status is set as Deleted,it is not visible in the IFS Cloud Web during this release as the Deleted Data Source is removed once the process is completed. This feature will be enhanced in the upcoming releases.

Error during the Deletion process

If any errors are encountered during the Deletion process,

  • The Refresh History will be updated as Error and logs can be viewed to identify further details.

  • The Mark For Deletion flag will be set to false.

  • Delete option is enabled.