Loop/Data Collection: Difference between revisions

Loop/Data Collection (view source)

9 bytes removed , 15 August 2014

Confirmed users

632

edits

Revision as of 20:24, 15 August 2014 (view source) AdamRoach (talk \| contribs) (→‎Upload URL Generation) ← Older edit		Revision as of 20:25, 15 August 2014 (view source) AdamRoach (talk \| contribs) (→‎Data Purging) Newer edit →
Line 354:		Line 354:

	=== Data Purging ===		=== Data Purging ===
	At the end of each ~~month~~, a data retention job will locate containers with a name indicating that they exceed the data retention policy for these reports (~~proposal: 6 months~~), and remove the containers (including all contained reports). This job also removes corresponding data from the index table. (tbd: table removal can probably be optimized through the use of partitions -- I need to look into Azure table storage a bit more).		At the end of each week, a data retention job will locate containers with a name indicating that they exceed the data retention policy for these reports (five weeks), and remove the containers (including all contained reports). This job also removes corresponding data from the index table. (tbd: table removal can probably be optimized through the use of partitions -- I need to look into Azure table storage a bit more).