Advanced Spark Meetup Recap

SVDS held our first public meetup on October 7th, hosting the Advanced Spark Meetup at our headquarters in Mountain View. To date, we’ve used Spark on five client projects, from ingestion pipelines to PySpark-enabled analytics platforms, using Java, Scala, and Python, and are always keen to dissect technical details with other big data practitioners. (We even have a contributor—high five, Andrew!) Our audience of engineers got right into the guts of Spark’s GraySort benchmark win last year with Chris Fregly from IBM Spark Technology Center. Here are a few highlights from the meetup. Our takeaways Configuration, configuration, configuration Getting the best performance out of Spark when every byte matters means digging into the guts of Spark and tuning parameters. Chris spent a large portion of the talk going over the…