This white paper provides an overview of how different NDT techniques can be modeled and simulated, highlighting the need for modern CAE tools that enable an efficient exploration of all variables involved.

November 9, 2018

Because of the scale of big data clusters, it is crucial that developers make the best use of a cluster’s hardware resources. However, it is challenging to figure out the best parameter settings for an entire big data software stack.

Intel and Dell EMC have collaborated on research to help developers better optimize big data clusters. Their research shows that the workload performance is CPU-sensitive and sensitive to scaling the number of nodes in a cluster. Accurate simulations of these workloads provide a development tool for choosing better values for configuration parameters.

The project compared the optimized parameter values suggested by Intel® CoFluent™ Technology for Big Data to the settings chosen by big-data experts. Results showed that Intel CoFluent delivered a 32% gain in the benchmark performance score over the parameter choices of expert human developers. This is an improvement equivalent to the performance gain typically seen from a new processor generation.