Advertising

On Friday, 16 January 2015 01:21:04 UTC+5:30, Feng Xiao wrote:
>
>
>
> On Tue Jan 13 2015 at 3:17:26 PM Arjun Satish <arjun....@gmail.com
> <javascript:>> wrote:
>
>> Since this feature is never going to be exposed, what advice do you have
>> for people who are using this feature and want to migrate to v3? Also, what
>> can we do as users to skip encoding certain fields with the new library?
>>
> We are aware of the migration difficulties from v2 to v3 due to removed
> features. As such we do not encourage existing users to migrate and commit
> to continue the support for proto2 syntax.
>
> You could have both proto2 and proto3 syntax files in your project and
> they are allowed to import each other despite the syntax differences.
>
>
>>
>> Thanks,
>>
>> On Tue, Jan 13, 2015 at 3:10 PM, Feng Xiao <xiao...@google.com
>> <javascript:>> wrote:
>>
>>>
>>>
>>> On Tue, Jan 13, 2015 at 3:05 PM, Arjun Satish <arjun....@gmail.com
>>> <javascript:>> wrote:
>>>
>>>> Feng,
>>>>
>>>> What do you mean when you say "In C++/Java/Python where we support both
>>>> proto2 and proto3, default values will continue to exist"?
>>>>
>>> What I meant is that you can still find its traces in the implementation
>>> but the feature itself is not exposed publicly in proto3 (i.e., we are
>>> still using it under-the-hood).
>>>
>>>
>>>> When I run protoc (v3) with the syntax="proto3" tag, it shows an error
>>>> "Explicit default values are not allowed in proto3." and exits (no code is
>>>> generated). This does not let me use other proto 3 features if my proto
>>>> definition file contain default values.
>>>>
>>>> Thanks for your timely responses! Highly appreciate it!
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jan 13, 2015 at 10:35 AM, Feng Xiao <xiao...@google.com
>>>> <javascript:>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Jan 12, 2015 at 10:40 PM, Arjun Satish <arjun....@gmail.com
>>>>> <javascript:>> wrote:
>>>>>
>>>>>> Would it be possible to re-introduce this feature in a subsequent
>>>>>> release? It seems like you are still using it under-the-hood.
>>>>>>
>>>>> In C++/Java/Python where we support both proto2 and proto3, default
>>>>> values will continue to exist. In new languages (e.g., ruby) though, the
>>>>> support for non-zero default values will be dropped completely.
>>>>>
>>>>>
>>>>>> And because of the benefits I mentioned above, I strongly feel that
>>>>>> it will only help the community.
>>>>>>
>>>>> As far as I know, the decision is final. Internally a lot Google
>>>>> projects have already adopted the new syntax and so far we have not heard
>>>>> problems caused by disallowing default values. It's unlikely this will be
>>>>> changed in the future. The omission of this feature (and other features)
>>>>> is
>>>>> to make the language simpler and to allow more idiomatic implementations
>>>>> in
>>>>> a wider range of languages. It's believed this decision will help the
>>>>> protobuf community (both protobuf maintainers and protobuf users) and we
>>>>> expect proto3 to be a version that can be more easily adopted than proto2
>>>>> by new users due to these simplifications. For existing users who rely on
>>>>> removed features, they can continue to use proto2 and that will be
>>>>> supported for a long time (if not forever). Currently we generally do not
>>>>> recommend migrating existing proto2 projects to proto3 because of
>>>>> incompatibility issues (e.g., extensions are dropped in proto3) and only
>>>>> recommend new users to use proto3.
>>>>>
>>>>>
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>> On Wed, Jan 7, 2015 at 8:01 PM, Feng Xiao <xiao...@google.com
>>>>>> <javascript:>> wrote:
>>>>>>
>>>>>>> +liujisi, who should have a better idea of why default value is
>>>>>>> dropped from proto3 and what alternatives users can rely on.
>>>>>>>
>>>>>>> Internally the design of proto3 has been discussed among a group of
>>>>>>> people for quite a long time, but most of them haven't subscribed this
>>>>>>> forum though...
>>>>>>> On Wed, Jan 7, 2015 at 18:52 Arjun Satish <arjun....@gmail.com
>>>>>>> <javascript:>> wrote:
>>>>>>>
>>>>>>>> Did anyone get a chance to look at this request?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wednesday, December 17, 2014 3:54:12 PM UTC-8, Arjun Satish
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hey guys,
>>>>>>>>>
>>>>>>>>> Thanks for all the hard work!
>>>>>>>>>
>>>>>>>>> I have a question regarding the decision to drop support for
>>>>>>>>> default values. Fields which are set to their default values are not
>>>>>>>>> serialized. I noticed that in the new code (3.0.0-alpha-1 for Java),
>>>>>>>>> this
>>>>>>>>> condition still holds true. But the default values used are the
>>>>>>>>> standard
>>>>>>>>> ones (0 for int64/int32 etc) and cannot be specified in the .proto
>>>>>>>>> file. In
>>>>>>>>> some of my code, I had reasons to use non-zero default values (-1 for
>>>>>>>>> some
>>>>>>>>> integers, 1024 for some others, 3.14 for some doubles etc). Using the
>>>>>>>>> old
>>>>>>>>> protocol buffers, this was trivial to implement. This was a great
>>>>>>>>> feature
>>>>>>>>> as we could save atleast 2 bytes for every "untouched" field (which
>>>>>>>>> comes
>>>>>>>>> in handy when we persist the data :-)).
>>>>>>>>>
>>>>>>>>> Is there any way we can retain specification of default values in
>>>>>>>>> the .proto files and using them in the generated encoders/decoders?
>>>>>>>>>
>>>>>>>>> Thanks very much!
>>>>>>>>>
>>>>>>>>> Looking forward to the 3.0 release!
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Arjun Satish
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wednesday, December 10, 2014 8:51:01 PM UTC-8, Feng Xiao wrote:
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I just published protobuf v3.0.0-alpha-1 on our github site:
>>>>>>>>>> https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-1
>>>>>>>>>>
>>>>>>>>>> This is the first alpha release of protobuf v3.0.0. In protobuf
>>>>>>>>>> v3.0.0, we will add a new protobuf language version (aka proto3) and
>>>>>>>>>> support a wider range of programming languages (to name a few: ruby,
>>>>>>>>>> php,
>>>>>>>>>> node.js, objective-c). This alpha version contains C++ and Java
>>>>>>>>>> implementation with partial proto3 support (see below for details).
>>>>>>>>>> In
>>>>>>>>>> future releases we will add support for more programming languages
>>>>>>>>>> and
>>>>>>>>>> implement the full proto3 feature set. Besides proto3, this alpha
>>>>>>>>>> version
>>>>>>>>>> also includes two other new features: map fields and arena
>>>>>>>>>> allocation. They
>>>>>>>>>> are implemented for both proto3 and the old protobuf language
>>>>>>>>>> version (aka
>>>>>>>>>> proto2).
>>>>>>>>>>
>>>>>>>>>> We are currently working on the documentation of these new
>>>>>>>>>> features and when it's ready it will be updated to our protobuf
>>>>>>>>>> developer guide
>>>>>>>>>> <https://developers.google.com/protocol-buffers/docs/overview>.
>>>>>>>>>> For the time being if you have any questions regarding proto3 or
>>>>>>>>>> other new
>>>>>>>>>> features, please post your question in the discussion group.
>>>>>>>>>>
>>>>>>>>>> CHANGS
>>>>>>>>>> =======
>>>>>>>>>> Version 3.0.0-alpha-1 (C++/Java):
>>>>>>>>>>
>>>>>>>>>> General
>>>>>>>>>> * Introduced Protocol Buffers language version 3 (aka proto3).
>>>>>>>>>>
>>>>>>>>>> When protobuf was initially opensourced it implemented
>>>>>>>>>> Protocol Buffers
>>>>>>>>>> language version 2 (aka proto2), which is why the version
>>>>>>>>>> number
>>>>>>>>>> started from v2.0.0. From v3.0.0, a new language version
>>>>>>>>>> (proto3) is
>>>>>>>>>> introduced while the old version (proto2) will continue to be
>>>>>>>>>> supported.
>>>>>>>>>>
>>>>>>>>>> The main intent of introducing proto3 is to clean up protobuf
>>>>>>>>>> before
>>>>>>>>>> pushing the language as the foundation of Google's new API
>>>>>>>>>> platform.
>>>>>>>>>> In proto3, the language is simplified, both for ease of use
>>>>>>>>>> and to
>>>>>>>>>> make it available in a wider range of programming languages.
>>>>>>>>>> At the
>>>>>>>>>> same time a few features are added to better support common
>>>>>>>>>> idioms
>>>>>>>>>> found in APIs.
>>>>>>>>>>
>>>>>>>>>> The following are the main new features in language version 3:
>>>>>>>>>>
>>>>>>>>>> 1. Removal of field presence logic for primitive value
>>>>>>>>>> fields, removal
>>>>>>>>>> of required fields, and removal of default values. This
>>>>>>>>>> makes proto3
>>>>>>>>>> significantly easier to implement with open struct
>>>>>>>>>> representations,
>>>>>>>>>> as in languages like Android Java, Objective C, or Go.
>>>>>>>>>> 2. Removal of unknown fields.
>>>>>>>>>> 3. Removal of extensions, which are instead replaced by a
>>>>>>>>>> new standard
>>>>>>>>>> type called Any.
>>>>>>>>>> 4. Fix semantics for unknown enum values.
>>>>>>>>>> 5. Addition of maps.
>>>>>>>>>> 6. Addition of a small set of standard types for
>>>>>>>>>> representation of time,
>>>>>>>>>> dynamic data, etc.
>>>>>>>>>> 7. A well-defined encoding in JSON as an alternative to
>>>>>>>>>> binary proto
>>>>>>>>>> encoding.
>>>>>>>>>>
>>>>>>>>>> This release (v3.0.0-alpha-1) includes partial proto3 support
>>>>>>>>>> for C++ and
>>>>>>>>>> Java. Items 6 (well-known types) and 7 (JSON format) in the
>>>>>>>>>> above feature
>>>>>>>>>> list are not implemented.
>>>>>>>>>>
>>>>>>>>>> A new notion "syntax" is introduced to specify whether a
>>>>>>>>>> .proto file
>>>>>>>>>> uses proto2 or proto3:
>>>>>>>>>>
>>>>>>>>>> // foo.proto
>>>>>>>>>> syntax = "proto3";
>>>>>>>>>> message Bar {...}
>>>>>>>>>>
>>>>>>>>>> If omitted, the protocol compiler will generate a warning and
>>>>>>>>>> "proto2" will
>>>>>>>>>> be used as the default. This warning will be turned into an
>>>>>>>>>> error in a
>>>>>>>>>> future release.
>>>>>>>>>>
>>>>>>>>>> We recommend that new Protocol Buffers users use proto3.
>>>>>>>>>> However, we do not
>>>>>>>>>> generally recommend that existing users migrate from proto2
>>>>>>>>>> from proto3 due
>>>>>>>>>> to API incompatibility, and we will continue to support
>>>>>>>>>> proto2 for a long
>>>>>>>>>> time.
>>>>>>>>>>
>>>>>>>>>> * Added support for map fields (implemented in C++/Java for
>>>>>>>>>> both proto2 and
>>>>>>>>>> proto3).
>>>>>>>>>>
>>>>>>>>>> Map fields can be declared using the following syntax:
>>>>>>>>>>
>>>>>>>>>> message Foo {
>>>>>>>>>> map<string, string> values = 1;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Data of a map field will be stored in memory as an unordered
>>>>>>>>>> map and it
>>>>>>>>>> can be accessed through generated accessors.
>>>>>>>>>>
>>>>>>>>>> C++
>>>>>>>>>> * Added arena allocation support (for both proto2 and proto3).
>>>>>>>>>>
>>>>>>>>>> Profiling shows memory allocation and deallocation
>>>>>>>>>> constitutes a significant
>>>>>>>>>> fraction of CPU-time spent in protobuf code and arena
>>>>>>>>>> allocation is a
>>>>>>>>>> technique introduced to reduce this cost. With arena
>>>>>>>>>> allocation, new
>>>>>>>>>> objects will be allocated from a large piece of preallocated
>>>>>>>>>> memory and
>>>>>>>>>> deallocation of these objects is almost free. Early adoption
>>>>>>>>>> shows 20% to
>>>>>>>>>> 50% improvement in some Google binaries.
>>>>>>>>>>
>>>>>>>>>> To enable arena support, add the following option to your
>>>>>>>>>> .proto file:
>>>>>>>>>>
>>>>>>>>>> option cc_enable_arenas = true;
>>>>>>>>>>
>>>>>>>>>> Protocol compiler will generate additional code to make the
>>>>>>>>>> generated
>>>>>>>>>> message classes work with arenas. This does not change the
>>>>>>>>>> existing API
>>>>>>>>>> of protobuf messages and does not affect wire format. Your
>>>>>>>>>> existing code
>>>>>>>>>> should continue to work after adding this option. In the
>>>>>>>>>> future we will
>>>>>>>>>> make this option enabled by default.
>>>>>>>>>>
>>>>>>>>>> To actually take advantage of arena allocation, you need to
>>>>>>>>>> use the arena
>>>>>>>>>> APIs when creating messages. A quick example of using the
>>>>>>>>>> arena API:
>>>>>>>>>>
>>>>>>>>>> {
>>>>>>>>>> google::protobuf::Arena arena;
>>>>>>>>>> // Allocate a protobuf message in the arena.
>>>>>>>>>> MyMessage* message = Arena::CreateMessage<
>>>>>>>>>> MyMessage>(&arena);
>>>>>>>>>> // All submessages will be allocated in the same arena.
>>>>>>>>>> if (!message->ParseFromString(data)) {
>>>>>>>>>> // Deal with malformed input data.
>>>>>>>>>> }
>>>>>>>>>> // Must not delete the message here. It will be deleted
>>>>>>>>>> automatically
>>>>>>>>>> // when the arena is destroyed.
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Currently arena does not work with map fields. Enabling arena
>>>>>>>>>> in a .proto
>>>>>>>>>> file containing map fields will result in compile errors in
>>>>>>>>>> the generated
>>>>>>>>>> code. This will be addressed in a future release.
>>>>>>>>>> =======
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Feng
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Protocol Buffers" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to protobuf+u...@googlegroups.com <javascript:>.
>>>>>>>> To post to this group, send email to prot...@googlegroups.com
>>>>>>>> <javascript:>.
>>>>>>>> Visit this group at http://groups.google.com/group/protobuf.
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.