Hadoop QA
added a comment - 05/Mar/15 06:57 -1 overall . Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12702701/HBASE-12466-v1.patch
against master branch at commit 0bdab85b065bd0876152ac30c2ec6d08adae8006.
ATTACHMENT ID: 12702701
+1 @author . The patch does not contain any @author tags.
+0 tests included . The patch appears to be a documentation patch that doesn't require tests.
+1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0)
+1 javac . The applied patch does not increase the total number of javac compiler warnings.
+1 javac . The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc . The javadoc tool did not generate any warning messages.
+1 checkstyle . The applied patch does not increase the total number of checkstyle errors
+1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings.
-1 release audit . The applied patch generated 1 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths . The patch introduces the following lines longer than 100:
+When you issue a Scan or Get, HBase uses your default set of authorizations to filter out cells that you do not have access to. A superuser can set the default set of authorizations for a given user by using the `set_auths` HBase Shell command or the link: http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/security/visibility/VisibilityClient.htmlsgetAuths%28org.apache.hadoop.conf.Configuration,%20java.lang.String%29[setAuths( )] method.
+You can specify a different authorization during the Scan or Get, by passing the AUTHORIZATIONS option in HBase Shell, or the link: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setAuthorizations%28org.apache.hadoop.hbase.security.visibility.Authorizations%29[setAuthorizations( )] method if you use the API. This authorization will be combined with your default set as an additional filter. It will further filter your results, rather than giving you additional authorization.
+You can specify a custom plugin or plugins by using the property `hbase.regionserver.scan.visibility.label.generator.class`. The output for the first `ScanLabelGenerator` will be the input for the next one, until the end of the list.
+The default implementation, which was implemented in link: https://issues.apache.org/jira/browse/HBASE-12466[HBASE-12466 ], loads two plugins, `FeedUserAuthScanLabelGenerator` and `DefinedSetFilterScanLabelGenerator`. See <<reading_cells_with_labels>>.
-1 site . The patch appears to cause mvn site goal to fail.
-1 core tests . The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
-1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.coprocessor.TestMasterObserver.testTableOperations(TestMasterObserver.java:1221)
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//artifact/patchprocess/checkstyle-aggregate.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13091//console
This message is automatically generated.

There is no more 'org.apache.hadoop.hbase.security.visibility.DefaultScanLabelGenerator'. It has been changed/removed. See the following comment in the org.apache.hadoop.hbase.security.visibility.VisibilityUtils#getScanLabelGenerators(). Also there is a mismatch between the JIRA link and the real real JIRA number.

// If no SLG is specified in conf, by default we'll add two SLGs
// 1. FeedUserAuthScanLabelGenerator
// 2. DefinedSetFilterScanLabelGenerator
// This stacking will achieve the following default behavior:
// 1. If there is no Auths in the scan, we will obtain the global defined set for the user
// from the labels table.
// 2. If there is Auths in the scan, we will examine the passed in Auths and filter out the
// labels that the user is not entitled to. Then use the resulting label set.
if (slgs.isEmpty()) {
slgs.add(ReflectionUtils.newInstance(FeedUserAuthScanLabelGenerator.class, conf));
slgs.add(ReflectionUtils.newInstance(DefinedSetFilterScanLabelGenerator.class, conf));
}
return slgs;

Jerry He
added a comment - 02/Mar/15 18:17 Hi, Misty Stanley-Jones
A superuser can see the default set of authorizations for a given user by using the `set_auths` HBase Shell command or the link: http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/security/visibility/VisibilityClient.htmlsgetAuths%28org.apache.hadoop.conf.Configuration,%20java.lang.String%29[setAuths( )] method.
There is a mismatch of the link (setAuths) and the method (get). Also do you mean "A superuser can set ..."?
+The default implementation class is `org.apache.hadoop.hbase.security.visibility.DefaultScanLabelGenerator`, which was implemented in link: https://issues.apache.org/jira/browse/HBASE-12466[HBASE-12468 ].
There is no more 'org.apache.hadoop.hbase.security.visibility.DefaultScanLabelGenerator'. It has been changed/removed. See the following comment in the org.apache.hadoop.hbase.security.visibility.VisibilityUtils#getScanLabelGenerators(). Also there is a mismatch between the JIRA link and the real real JIRA number.
// If no SLG is specified in conf, by default we'll add two SLGs
// 1. FeedUserAuthScanLabelGenerator
// 2. DefinedSetFilterScanLabelGenerator
// This stacking will achieve the following default behavior:
// 1. If there is no Auths in the scan, we will obtain the global defined set for the user
// from the labels table.
// 2. If there is Auths in the scan, we will examine the passed in Auths and filter out the
// labels that the user is not entitled to. Then use the resulting label set.
if (slgs.isEmpty()) {
slgs.add(ReflectionUtils.newInstance(FeedUserAuthScanLabelGenerator.class, conf));
slgs.add(ReflectionUtils.newInstance(DefinedSetFilterScanLabelGenerator.class, conf));
}
return slgs;

Hadoop QA
added a comment - 02/Mar/15 03:56 -1 overall . Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12701761/HBASE-12466.patch
against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
ATTACHMENT ID: 12701761
+1 @author . The patch does not contain any @author tags.
+0 tests included . The patch appears to be a documentation patch that doesn't require tests.
+1 hadoop versions . The patch compiles with all supported hadoop versions (2.4.1 2.5.2 2.6.0)
+1 javac . The applied patch does not increase the total number of javac compiler warnings.
+1 javac . The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc . The javadoc tool did not generate any warning messages.
+1 checkstyle . The applied patch does not increase the total number of checkstyle errors
+1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings.
+1 release audit . The applied patch does not increase the total number of release audit warnings.
-1 lineLengths . The patch introduces the following lines longer than 100:
+When you issue a Scan or Get, HBase uses your default set of authorizations to filter out cells that you do not have access to. A superuser can see the default set of authorizations for a given user by using the `set_auths` HBase Shell command or the link: http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/security/visibility/VisibilityClient.htmlsgetAuths%28org.apache.hadoop.conf.Configuration,%20java.lang.String%29[setAuths( )] method.
+You can specify a different authorization during the Scan or Get, by passing the AUTHORIZATIONS option in HBase Shell, or the link: http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setAuthorizations%28org.apache.hadoop.hbase.security.visibility.Authorizations%29[setAuthorizations( )] method if you use the API. This authorization will be combined with your default set as an additional filter. It will further filter your results, rather than giving you additional authorization.
+The default implementation class is `org.apache.hadoop.hbase.security.visibility.DefaultScanLabelGenerator`, which was implemented in link: https://issues.apache.org/jira/browse/HBASE-12466[HBASE-12468 ]. See <<reading_cells_with_labels>>.
+You can also configure a set, or "stack", of `ScanLabelGenerator`s to be used by the system, as a comma-separated list. The output for the first `ScanLabelGenerator` will be the input for the next one, until the end of the list.
+1 site . The mvn site goal succeeds with this patch.
-1 core tests . The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
-1 core zombie tests . There are 1 zombie test(s): at org.apache.hadoop.hbase.TestHColumnDescriptorDefaultVersions.testCreateTableWithDefaultFromConf(TestHColumnDescriptorDefaultVersions.java:104)
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/checkstyle-aggregate.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13026//console
This message is automatically generated.

It is documented in the book: section 57.3.3. Implementing Your Own Visibility Label Algorithm
That is the section you may want to update.
For the details, you can refer to the org.apache.hadoop.hbase.security.visibility.VisibilityUtils class.
The comments in the method getScanLabelGenerators() explains a lot.

Jerry He
added a comment - 26/Feb/15 20:10 Hi, Misty Stanley-Jones
It is documented in the book: section 57.3.3. Implementing Your Own Visibility Label Algorithm
That is the section you may want to update.
For the details, you can refer to the org.apache.hadoop.hbase.security.visibility.VisibilityUtils class.
The comments in the method getScanLabelGenerators() explains a lot.

You can use ScanLabelGenerator to control how authorization labels are fetched and filtered during the authorization process.
HBase have ScanLabelGenerator implementations to provide the default authorization behavior (documented in HBASE-12468).
You can also provide your own implementation of the interface ScanLabelGenerator.
ScanLabelGenerators can also be stacked to provide the desired behavior.
The output from one ScanLabelGenerator will be the input for the next ScanLabelGenerator in the stack.

Jerry He
added a comment - 24/Feb/15 18:26 Hi, Misty Stanley-Jones
Here is a draft
You can use ScanLabelGenerator to control how authorization labels are fetched and filtered during the authorization process.
HBase have ScanLabelGenerator implementations to provide the default authorization behavior (documented in HBASE-12468 ).
You can also provide your own implementation of the interface ScanLabelGenerator.
ScanLabelGenerators can also be stacked to provide the desired behavior.
The output from one ScanLabelGenerator will be the input for the next ScanLabelGenerator in the stack.