hadoop-hdfs-user mailing list archives

Best tests for CPU or network bounded executions in hadoop-gridmix-2.6.0.jar

Date

Tue, 24 Mar 2015 18:00:04 GMT

Hi,
I am trying to see what are the current tests that the Gridmix2 package
do in MapReduce2.x. I see that the Gridmix jar
|hadoop-gridmix-2.6.0.jar|does not have anymore the WebDataSort,
WebDataScan, MonsterQuery, Combiner, and Streaming tests. The current
tests are in [1].
From all of these tests, what is the best test that is CPU bounded
(like the PI calculation), and the best test that is network bounded
(like the wordcount)?
When I say CPU bounded tests, I say the tests that need lots of CPU
calculation, and, for the network bounded, I mean the jobs that needs to
transfer lots of data between mappers and reducers.
[1]
|~/Programs/hadoop/share/hadoop/tools/sources$ tree org/apache/hadoop/mapred/
org/apache/hadoop/mapred/
└── gridmix
├── CommonJobTest.java
├── DebugJobFactory.java
├── DebugJobProducer.java
├── DummyResourceCalculatorPlugin.java
├── GridmixTestUtils.java
├── TestCompressionEmulationUtils.java
├── TestDistCacheEmulation.java
├── TestFilePool.java
├── TestFileQueue.java
├── TestGridMixClasses.java
├── TestGridmixMemoryEmulation.java
├── TestGridmixRecord.java
├── TestGridmixStatistics.java
├── TestGridmixSubmission.java
├── TestGridmixSummary.java
├── TestHighRamJob.java
├── TestLoadJob.java
├── TestPseudoLocalFs.java
├── TestRandomAlgorithm.java
├── TestRandomTextDataGenerator.java
├── TestRecordFactory.java
├── TestResourceUsageEmulators.java
├── TestSleepJob.java
└── TestUserResolve.java
|
​
--
--
Thanks,