Test Driving Spark SQL with Intel SGX on Encrypted Data

Protecting sensitive business and personal information is a central requirement when enterprises move to the cloud. Many aspects of this requirement are already handled at various levels. Data-at-rest can be secured in cloud stores by encrypting it before storage, while data-in-flight is transmitted on protected channels such as TLS and HTTPS. Data-in-use, processed in cloud compute nodes, is kept in isolated virtual machines or containers. And now, a new generation of secure hardware technology (Intel SGX, AMD SME/SEV) provides additional protection of data processing in public clouds, by making the process memory inaccessible to malicious privileged software or system administrators…