hive-dev mailing list archives

[jira] [Commented] (HIVE-2282) Local mode needs to work well with block sampling

Date

Fri, 22 Jul 2011 17:41:00 GMT

[ https://issues.apache.org/jira/browse/HIVE-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069635#comment-13069635
]
jiraposter@reviews.apache.org commented on HIVE-2282:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1132/
-----------------------------------------------------------
(Updated 2011-07-22 17:40:44.736466)
Review request for hive and Siying Dong.
Changes
-------
I added the q.out file which I had forgotten for the new q file.
I also modified the test queries to select count(1) instead of selecting keys and values.
Summary
-------
A query should run in local mode when block sampling is used and the sample is small enough.
The size of the sample is currently being estimated, as it is done to estimate the number
of reducers.
This addresses bug HIVE-2282.
https://issues.apache.org/jira/browse/HIVE-2282
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 53769a0
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cd3de76
ql/src/test/org/apache/hadoop/hive/ql/hooks/VerifyIsLocalModeHook.java PRE-CREATION
ql/src/test/queries/clientpositive/sample_islocalmode_hook.q PRE-CREATION
ql/src/test/results/clientpositive/sample_islocalmode_hook.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1132/diff
Testing
-------
TestCliDriver TestNegativeCliDriver, manually tested
Thanks,
Kevin
> Local mode needs to work well with block sampling
> -------------------------------------------------
>
> Key: HIVE-2282
> URL: https://issues.apache.org/jira/browse/HIVE-2282
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2282.1.patch.txt, HIVE-2282.2.patch.txt, HIVE-2282.3.patch.txt
>
>
> Currently, if block sampling is enabled and large set of data are sampled to a small
set, local mode needs to be kicked in.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira