[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908018#comment-16908018
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: Support empty Parquet
files creation
URL: https://github.com/apache/drill/pull/1836#issuecomment-521618404
Ran all tests on the test cluster, all passed. LGTM, +1
@oleg-zinovev thanks for making the changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907298#comment-16907298
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: Support empty Parquet
files creation
URL: https://github.com/apache/drill/pull/1836#issuecomment-521263751
@oleg-zinovev thanks for making the changes, a couple of minor comments are
left...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907153#comment-16907153
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: Support empty Parquet
files creation
URL: https://github.com/apache/drill/pull/1836#issuecomment-521191646
> @arina-ielchiieva
> Some time spent debugging the test showed that the last schema contains
all fields. The field is added in ProjectRecordBatch#setupNewSchemaFromInput.
> In the original version of the test, field A was not added due to plan
optimization - condition `1=0` was replaced by` limit 0`
>
> I can still provide a solution with combining schema if required.
This case we don't need schema merge,
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907085#comment-16907085
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: Support empty Parquet files
creation
URL: https://github.com/apache/drill/pull/1836#issuecomment-521174404
@arina-ielchiieva
Some time spent debugging the test showed that the last schema contains all
fields. The field is added in ProjectRecordBatch#setupNewSchemaFromInput.
In the original version of the test, field A was not added due to plan
optimization - condition `1=0` was replaced by` limit 0`
I can still provide a solution with combining schema if required.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906235#comment-16906235
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520813625
Good questions, you can investigate how now union types are handled when
there are data. Regarding who wins, maybe you can look into
`org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch` to see how it
creates combined schema using precedence rules.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906160#comment-16906160
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520819862
> Good questions, you can investigate how now union types are handled.
Regarding who wins, maybe you can look into
`org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch` to see how it
Cretes combined schema using precedence rules.
@arina-ielchiieva , thanks for your advice.
I will try to make a combined scheme by the end of the week.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906140#comment-16906140
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520813625
Good questions, you can investigate how now union types are handled when
there are data. Regarding who wins, maybe you can look into
`org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch` to see how it
Cretes combined schema using precedence rules.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906139#comment-16906139
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520813625
Good questions, you can investigate how now union types are handled.
Regarding who wins, maybe you can look into
`org.apache.drill.exec.physical.impl.union.UnionAllRecordBatch` to see how it
Cretes combined schema using precedence rules.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906134#comment-16906134
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520812579
> Is there a way to write combined schema in this case?
@arina-ielchiieva , thank you for your comment.
I can try to make a combined scheme, but:
- What type of field should be written if the first BatchSchema contains
field "A" with type "bigint", and the second - field "A" with type "varchar"?
The last one wins?
- What about union vectors?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906133#comment-16906133
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520812579
> Please provide example...
> Is there a way to write combined schema in this case?
@arina-ielchiieva , thank you for your comment.
I can try to make a combined scheme, but:
- What type of field should be written if the first BatchSchema contains
field "A" with type "bigint", and the second - field "A" with type "varchar"?
The last one wins?
- What about union vectors?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906127#comment-16906127
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520807379
Is there a way to write combined schema in this case?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906126#comment-16906126
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520807005
> Please provide example...
I added a test TestParquetWriterEmptyFiles#testWriteEmptySchemaChange. As
you can see, there is no "a" field in the written schema.
Probably, it would be correct to write schema for all empty scans, but it
will lead to writing "garbage" empty parquet files, if the scan with data is at
the end of batch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906093#comment-16906093
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520795098
> What behavior will be in this case? Failure? No-op?
Remained unchanged (No-op)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906091#comment-16906091
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520795098
> What behavior will be in this case? Failure? No-op?
Remained unchanged (No-op)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906088#comment-16906088
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520794418
> Ignores all schemas except last while writing empty parquet file
Please provide example...
> Not support empty schemas (e.g. create table .. as select * from
empty.json, e.g. {})
What behavior will be in this case? Failure? No-op?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906089#comment-16906089
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520794418
> Ignores all schemas except last while writing empty parquet file
Please provide example...
> Not support empty schemas (e.g. create table .. as select * from
empty.json, e.g. {})
What behavior will be in this case? Failure? No-op?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906087#comment-16906087
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520794009
Regarding this comment
> Questions:
TestParquetWriterEmptyFiles#testMultipleWriters now creates several empty
files, but not fails, since reading of empty parquet is supported. Should I
rewrite comment or remove the test?
I guess you can remove these tests and add new tests into
`org.apache.drill.exec.store.parquet.TestEmptyParquet`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906044#comment-16906044
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520776279
> @oleg-zinovev support empty parquet files reading is already merged into
master
([4f4e1af](https://github.com/apache/drill/commit/4f4e1af53c9abccd1996f3b6841731e68768b48e)).
Do you plan on working on adding support for writing empty parquet files? We
plan to include it in next Drill release (end of August / beginning of
September). If yes, please factor out writing empty parquet and update the PR.
Yes, I think I’ll do it today.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906037#comment-16906037
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-520774175
@oleg-zinovev support empty parquet files reading is already merged into
master
(https://github.com/apache/drill/commit/4f4e1af53c9abccd1996f3b6841731e68768b48e).
Do you plan on working on adding support for writing empty parquet files? We
plan to include it in next Drill release (end of August / beginning of
September). If yes, please factor out writing empty parquet and update the PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899989#comment-16899989
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-518187400
@oleg-zinovev thanks, I have assigned
https://issues.apache.org/jira/browse/DRILL-7156 to you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.16.0
>Reporter: Sayalee Bhanavase
>Assignee: Oleg Zinoviev
>Priority: Major
> Fix For: 1.17.0
>
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899985#comment-16899985
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-518186190
Ok, I'll try to rewrite the commit within a week.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899984#comment-16899984
]
ASF GitHub Bot commented on DRILL-7156:
---
arina-ielchiieva commented on issue #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836#issuecomment-518183790
@oleg-zinovev thanks for making the changes, though the situation is little
bit awkward, since I was working on similar changes and did not know you intend
to do them as well (https://issues.apache.org/jira/browse/DRILL-4517). Though I
was working on reading empty parquet files but not writing them. I suggest you
separate out writing empty parquet files into separate PR as for reading it
might be better if my changes will be used instead: first you change metadata
cache files and this would affect backward compatibility as well as will have
to store more information than needed, secondly your changes does not seem to
optimize reading complex types. What do you think?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899949#comment-16899949
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on issue #1836: DRILL-7156: empty parquet files support
URL: https://github.com/apache/drill/pull/1836#issuecomment-518161785
@arina-ielchiieva, could you please review?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899942#comment-16899942
]
ASF GitHub Bot commented on DRILL-7156:
---
oleg-zinovev commented on pull request #1836: DRILL-7156: empty parquet files
support
URL: https://github.com/apache/drill/pull/1836
PR for Drill empty parquet files read and write support.
Known limitations:
1) Not working for hive parquet for now
2) Ignores all schemas except last while writing empty parquet file
3) Not support empty schemas (e.g. create table `..` as select * from
`empty.json, e.g. {}`)
Short changes description:
1) Parquet footer metadata added
2) Parquet writer checks that at least 1 row has been written. If not -
creates a empty parquet file with footer.
3) EmptyParquetRowGroupScan and EmptyParquetScanBatchCreator added
Questions:
1) TestParquetWriterEmptyFiles#testMultipleWriters now creates several empty
files, but not fails, since reading of empty parquet is supported. Should I
rewrite comment or remove the test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856605#comment-16856605
]
Arina Ielchiieva commented on DRILL-7156:
-
[~le.louch] there are a similar issue in Drill:
https://issues.apache.org/jira/browse/DRILL-4517
https://issues.apache.org/jira/browse/DRILL-6885
If you can contribute the patch, it would be great.
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[
https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840295#comment-16840295
]
Oleg Zinoviev commented on DRILL-7156:
--
I had to deal with this problem. As a result, I made my own Apache Drill build,
which supports the creation and reading of empty parquet files. I can try to
prepare a patch in the main repository, if this is really a Drill problem.
> Empty Parquet is not getting created if 0 records in result
> ---
>
> Key: DRILL-7156
> URL: https://issues.apache.org/jira/browse/DRILL-7156
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Reporter: Sayalee Bhanavase
>Priority: Blocker
>
> I am creating parquet tables out of joins. If there is no record in join, it
> does not create empty. table and when I reused the table my further script
> fails.
> Has anyone faced this issue? Any suggestion or workaround?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)