1 PB of data, 136 Billion Rows and how we avoided paying $2730 per day
Andreas Scherbaum
EDB
This talk shows how we loaded a PB of data and the challenges we solved along the way.
We were challenged on whether WarehousePG can really handle a Petabyte of data. After all, the product claims "Petabyte Scale". So we started loading a PB into the database, the start of an interesting journey. We learned a lot about the performance and cost of operating a database in the cloud, before moving to bare metal servers. This process also presented challenges; sometimes, the network switch couldn't keep up.
1 PB of data, 136 Billion Rows and how we avoided paying $2730 per day
Andreas Scherbaum
EDB
This talk shows how we loaded a PB of data and the challenges we solved along the way.
We were challenged on whether WarehousePG can really handle a Petabyte of data. After all, the product claims "Petabyte Scale". So we started loading a PB into the database, the start of an interesting journey. We learned a lot about the performance and cost of operating a database in the cloud, before moving to bare metal servers. This process also presented challenges; sometimes, the network switch couldn't keep up.