A similar sort would take 5 hours if run against a well-known leader database.

The CDR table comes from a telecom with 6 million subscribers. Stored in a database, it weighs nearly 2 TB.

The starting CDR table is made up of 42 different files (in .gel_anatella format), that’s 1 billion rows or 42 days of data in total. Anatella sorts the CDR table on column A (which contains over 6 million different call numbers). This data transformation graph performs the sort on a laptop:

 

The ticket table contains all purchases made by over 1 million customers over 2 years at the finest scale (one row in the table represents the purchase of a product). We first extracted this ticket table out of Teradata (over 2TB in the database) to get a 450 GB .sas7bdat file. Then, using Anatella, we converted this .sas7bdat file to a more compact 23 GB cgel_anatella file. This new file was used as the basis for calculating the percentage of purchases on the web. This Anatella data transformation graph realizes the requested aggregate: