diff options
author | glider@chromium.org <glider@chromium.org@0039d316-1c4b-4281-b951-d872f2087c98> | 2013-08-12 08:59:46 +0000 |
---|---|---|
committer | glider@chromium.org <glider@chromium.org@0039d316-1c4b-4281-b951-d872f2087c98> | 2013-08-12 08:59:46 +0000 |
commit | 9f1702fcfd12d85625e4406702fc39716bf55bca (patch) | |
tree | f649d49cd44b4048a8cbc03c2a33063443671955 /third_party/protobuf | |
parent | 22cd8c9da08866edfa594b08d011d6dadfcd3d6c (diff) | |
download | chromium_src-9f1702fcfd12d85625e4406702fc39716bf55bca.zip chromium_src-9f1702fcfd12d85625e4406702fc39716bf55bca.tar.gz chromium_src-9f1702fcfd12d85625e4406702fc39716bf55bca.tar.bz2 |
Update protobuf to r428, part 1.
This has been done by:
- patching third_party/protobuf with the upstream diff for r423:426 and r428 (427 has already been cherrypicked)
- manually applying the rejected hunks in the following files:
src/google/protobuf/wire_format.cc
src/google/protobuf/wire_format_lite.h
src/google/protobuf/unknown_field_set.cc
src/google/protobuf/descriptor.pb.h
src/google/protobuf/extension_set.cc
src/google/protobuf/wire_format_lite.cc
src/google/protobuf/descriptor.pb.cc
src/google/protobuf/wire_format.h
src/google/protobuf/compiler/plugin.pb.h
src/google/protobuf/compiler/cpp/cpp_helpers.h
src/google/protobuf/compiler/cpp/cpp_file.cc
vsprojects/lite-test.vcproj.rej
vsprojects/libprotobuf-lite.vcproj.rej
vsprojects/libprotobuf.vcproj.rej
vsprojects/tests.vcproj.rej
vsprojects/extract_includes.bat.rej
- adding the new cc/h files to protobuf.gyp and disabling MSVC warning C4291
- rebuilding descriptor.pb.{cc,h} using the upstream protoc r425
- fixing compilation (replaced WireFormat with WireFormatLite where applicable)
- cherry-picking the upstream r505
- applying patch from https://code.google.com/p/protobuf/issues/detail?id=539
To reduce the changelist size, some of the test files were excluded.
They will be committed separately: https://codereview.chromium.org/21231003
BUG=259808
TBR=agl@chromium.org, pliard@google.com
Review URL: https://codereview.chromium.org/21208003
git-svn-id: svn://svn.chromium.org/chrome/trunk/src@216961 0039d316-1c4b-4281-b951-d872f2087c98
Diffstat (limited to 'third_party/protobuf')
248 files changed, 24693 insertions, 4010 deletions
diff --git a/third_party/protobuf/CHANGES.txt b/third_party/protobuf/CHANGES.txt index 737f865..a21e956 100644 --- a/third_party/protobuf/CHANGES.txt +++ b/third_party/protobuf/CHANGES.txt @@ -1,3 +1,61 @@ +2012-09-19 version 2.5.0: + + General + * New notion "import public" that allows a proto file to forward the content + it imports to its importers. For example, + // foo.proto + import public "bar.proto"; + import "baz.proto"; + + // qux.proto + import "foo.proto"; + // Stuff defined in bar.proto may be used in this file, but stuff from + // baz.proto may NOT be used without importing it explicitly. + This is useful for moving proto files. To move a proto file, just leave + a single "import public" in the old proto file. + * New enum option "allow_alias" that specifies whether different symbols can + be assigned the same numeric value. Default value is "true". Setting it to + false causes the compiler to reject enum definitions where multiple symbols + have the same numeric value. + + C++ + * New generated method set_allocated_foo(Type* foo) for message and string + fields. This method allows you to set the field to a pre-allocated object + and the containing message takes the ownership of that object. + * Added SetAllocatedExtension() and ReleaseExtension() to extensions API. + * Custom options are now formatted correctly when descriptors are printed in + text format. + * Various speed optimizations. + + Java + * Comments in proto files are now collected and put into generated code as + comments for corresponding classes and data members. + * Added Parser to parse directly into messages without a Builder. For + example, + Foo foo = Foo.getParser().ParseFrom(input); + Using Parser is ~25% faster than using Builder to parse messages. + * Added getters/setters to access the underlying ByteString of a string field + directly. + * ByteString now supports more operations: substring(), prepend(), and + append(). The implementation of ByteString uses a binary tree structure + to support these operations efficiently. + * New method findInitializationErrors() that lists all missing required + fields. + * Various code size and speed optimizations. + + Python + * Added support for dynamic message creation. DescriptorDatabase, + DescriptorPool, and MessageFactory work like their C++ couterparts to + simplify Descriptor construction from *DescriptorProtos, and MessageFactory + provides a message instance from a Descriptor. + * Added pickle support for protobuf messages. + * Unknown fields are now preserved after parsing. + * Fixed bug where custom options were not correctly populated. Custom + options can be accessed now. + * Added EnumTypeWrapper that provides better accessibility to enum types. + * Added ParseMessage(descriptor, bytes) to generate a new Message instance + from a descriptor and a byte string. + 2011-05-01 version 2.4.1: C++ diff --git a/third_party/protobuf/README.chromium b/third_party/protobuf/README.chromium index e15fcae..c8e411f 100644 --- a/third_party/protobuf/README.chromium +++ b/third_party/protobuf/README.chromium @@ -4,7 +4,7 @@ URL: http://protobuf.googlecode.com/svn/trunk License: BSD License File: COPYING.txt Version: unknown -Revision: r423 +Revision: r428 Security Critical: yes Local files (not taken from upstream): @@ -26,9 +26,7 @@ This code has been patched to make the target protobuf_lite a component so that targets that depend on it can be componentized. See http://crbug.com/172800 for details, and r179806 for the patch. -Revisions r427, r430, r475, r476 and 504 were cherry-picked from upstream. -Patch from http://code.google.com/p/protobuf/issues/detail?id=425 was -cherry-picked from upstream. +Revisions r430, r475, r476 and 504 were cherry-picked from upstream. Notes about Java: We have not forked the Java version of protobuf-lite, so the Java version does diff --git a/third_party/protobuf/java/pom.xml b/third_party/protobuf/java/pom.xml index e28d6db..7ec6d91 100644 --- a/third_party/protobuf/java/pom.xml +++ b/third_party/protobuf/java/pom.xml @@ -102,6 +102,7 @@ <arg value="--proto_path=src/test/java" /> <arg value="../src/google/protobuf/unittest.proto" /> <arg value="../src/google/protobuf/unittest_import.proto" /> + <arg value="../src/google/protobuf/unittest_import_public.proto" /> <arg value="../src/google/protobuf/unittest_mset.proto" /> <arg value="src/test/java/com/google/protobuf/multiple_files_test.proto" /> @@ -117,6 +118,7 @@ value="../src/google/protobuf/unittest_custom_options.proto" /> <arg value="../src/google/protobuf/unittest_lite.proto" /> <arg value="../src/google/protobuf/unittest_import_lite.proto" /> + <arg value="../src/google/protobuf/unittest_import_public_lite.proto" /> <arg value="../src/google/protobuf/unittest_lite_imports_nonlite.proto" /> <arg value="../src/google/protobuf/unittest_enormous_descriptor.proto" /> <arg value="../src/google/protobuf/unittest_no_generic_services.proto" /> @@ -158,6 +160,13 @@ <include>**/UninitializedMessageException.java</include> <include>**/UnmodifiableLazyStringList.java</include> <include>**/WireFormat.java</include> + <include>**/Parser.java</include> + <include>**/AbstractParser.java</include> + <include>**/BoundedByteString.java</include> + <include>**/LiteralByteString.java</include> + <include>**/RopeByteString.java</include> + <include>**/Utf8.java</include> + <include>**/LazyField.java</include> </includes> <testIncludes> <testInclude>**/LiteTest.java</testInclude> diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessage.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessage.java index b9d8301..f4d115d 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessage.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessage.java @@ -32,6 +32,7 @@ package com.google.protobuf; import com.google.protobuf.Descriptors.Descriptor; import com.google.protobuf.Descriptors.FieldDescriptor; +import com.google.protobuf.GeneratedMessage.ExtendableBuilder; import com.google.protobuf.Internal.EnumLite; import java.io.IOException; @@ -81,6 +82,25 @@ public abstract class AbstractMessage extends AbstractMessageLite return true; } + public List<String> findInitializationErrors() { + return Builder.findMissingFields(this); + } + + public String getInitializationErrorString() { + return delimitWithCommas(findInitializationErrors()); + } + + private static String delimitWithCommas(List<String> parts) { + StringBuilder result = new StringBuilder(); + for (String part : parts) { + if (result.length() > 0) { + result.append(", "); + } + result.append(part); + } + return result.toString(); + } + @Override public final String toString() { return TextFormat.printToString(this); @@ -209,6 +229,15 @@ public abstract class AbstractMessage extends AbstractMessageLite } /** + * Package private helper method for AbstractParser to create + * UninitializedMessageException with missing field information. + */ + @Override + UninitializedMessageException newUninitializedMessageException() { + return Builder.newUninitializedMessageException(this); + } + + /** * Helper method for implementing {@link Message#hashCode()}. * <p> * This is needed because {@link java.lang.Enum#hashCode()} is final, but we @@ -251,6 +280,14 @@ public abstract class AbstractMessage extends AbstractMessageLite return (BuilderType) this; } + public List<String> findInitializationErrors() { + return findMissingFields(this); + } + + public String getInitializationErrorString() { + return delimitWithCommas(findInitializationErrors()); + } + public BuilderType mergeFrom(final Message other) { if (other.getDescriptorForType() != getDescriptorForType()) { throw new IllegalArgumentException( @@ -314,7 +351,7 @@ public abstract class AbstractMessage extends AbstractMessageLite } if (!mergeFieldFrom(input, unknownFields, extensionRegistry, - this, tag)) { + getDescriptorForType(), this, null, tag)) { // end group tag break; } @@ -323,25 +360,93 @@ public abstract class AbstractMessage extends AbstractMessageLite return (BuilderType) this; } + /** helper method to handle {@code builder} and {@code extensions}. */ + private static void addRepeatedField( + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + FieldDescriptor field, + Object value) { + if (builder != null) { + builder.addRepeatedField(field, value); + } else { + extensions.addRepeatedField(field, value); + } + } + + /** helper method to handle {@code builder} and {@code extensions}. */ + private static void setField( + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + FieldDescriptor field, + Object value) { + if (builder != null) { + builder.setField(field, value); + } else { + extensions.setField(field, value); + } + } + + /** helper method to handle {@code builder} and {@code extensions}. */ + private static boolean hasOriginalMessage( + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + FieldDescriptor field) { + if (builder != null) { + return builder.hasField(field); + } else { + return extensions.hasField(field); + } + } + + /** helper method to handle {@code builder} and {@code extensions}. */ + private static Message getOriginalMessage( + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + FieldDescriptor field) { + if (builder != null) { + return (Message) builder.getField(field); + } else { + return (Message) extensions.getField(field); + } + } + + /** helper method to handle {@code builder} and {@code extensions}. */ + private static void mergeOriginalMessage( + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + FieldDescriptor field, + Message.Builder subBuilder) { + Message originalMessage = getOriginalMessage(builder, extensions, field); + if (originalMessage != null) { + subBuilder.mergeFrom(originalMessage); + } + } + /** - * Like {@link #mergeFrom(CodedInputStream, UnknownFieldSet.Builder, - * ExtensionRegistryLite, Message.Builder)}, but parses a single field. + * Like {@link #mergeFrom(CodedInputStream, ExtensionRegistryLite)}, but + * parses a single field. + * + * When {@code builder} is not null, the method will parse and merge the + * field into {@code builder}. Otherwise, it will try to parse the field + * into {@code extensions}, when it's called by the parsing constructor in + * generated classes. + * * Package-private because it is used by GeneratedMessage.ExtendableMessage. * @param tag The tag, which should have already been read. * @return {@code true} unless the tag is an end-group tag. */ static boolean mergeFieldFrom( - final CodedInputStream input, - final UnknownFieldSet.Builder unknownFields, - final ExtensionRegistryLite extensionRegistry, - final Message.Builder builder, - final int tag) throws IOException { - final Descriptor type = builder.getDescriptorForType(); - + CodedInputStream input, + UnknownFieldSet.Builder unknownFields, + ExtensionRegistryLite extensionRegistry, + Descriptor type, + Message.Builder builder, + FieldSet<FieldDescriptor> extensions, + int tag) throws IOException { if (type.getOptions().getMessageSetWireFormat() && tag == WireFormat.MESSAGE_SET_ITEM_TAG) { mergeMessageSetExtensionFromCodedStream( - input, unknownFields, extensionRegistry, builder); + input, unknownFields, extensionRegistry, type, builder, extensions); return true; } @@ -376,8 +481,10 @@ public abstract class AbstractMessage extends AbstractMessageLite } else { field = null; } - } else { + } else if (builder != null) { field = type.findFieldByNumber(fieldNumber); + } else { + field = null; } boolean unknown = false; @@ -413,13 +520,13 @@ public abstract class AbstractMessage extends AbstractMessageLite // enum, drop it (don't even add it to unknownFields). return true; } - builder.addRepeatedField(field, value); + addRepeatedField(builder, extensions, field, value); } } else { while (input.getBytesUntilLimit() > 0) { final Object value = FieldSet.readPrimitiveField(input, field.getLiteType()); - builder.addRepeatedField(field, value); + addRepeatedField(builder, extensions, field, value); } } input.popLimit(limit); @@ -434,10 +541,10 @@ public abstract class AbstractMessage extends AbstractMessageLite subBuilder = builder.newBuilderForField(field); } if (!field.isRepeated()) { - subBuilder.mergeFrom((Message) builder.getField(field)); + mergeOriginalMessage(builder, extensions, field, subBuilder); } input.readGroup(field.getNumber(), subBuilder, extensionRegistry); - value = subBuilder.build(); + value = subBuilder.buildPartial(); break; } case MESSAGE: { @@ -448,10 +555,10 @@ public abstract class AbstractMessage extends AbstractMessageLite subBuilder = builder.newBuilderForField(field); } if (!field.isRepeated()) { - subBuilder.mergeFrom((Message) builder.getField(field)); + mergeOriginalMessage(builder, extensions, field, subBuilder); } input.readMessage(subBuilder, extensionRegistry); - value = subBuilder.build(); + value = subBuilder.buildPartial(); break; } case ENUM: @@ -470,22 +577,28 @@ public abstract class AbstractMessage extends AbstractMessageLite } if (field.isRepeated()) { - builder.addRepeatedField(field, value); + addRepeatedField(builder, extensions, field, value); } else { - builder.setField(field, value); + setField(builder, extensions, field, value); } } return true; } - /** Called by {@code #mergeFieldFrom()} to parse a MessageSet extension. */ + /** + * Called by {@code #mergeFieldFrom()} to parse a MessageSet extension. + * If {@code builder} is not null, this method will merge MessageSet into + * the builder. Otherwise, it will merge the MessageSet into {@code + * extensions}. + */ private static void mergeMessageSetExtensionFromCodedStream( - final CodedInputStream input, - final UnknownFieldSet.Builder unknownFields, - final ExtensionRegistryLite extensionRegistry, - final Message.Builder builder) throws IOException { - final Descriptor type = builder.getDescriptorForType(); + CodedInputStream input, + UnknownFieldSet.Builder unknownFields, + ExtensionRegistryLite extensionRegistry, + Descriptor type, + Message.Builder builder, + FieldSet<FieldDescriptor> extensions) throws IOException { // The wire format for MessageSet is: // message MessageSet { @@ -504,10 +617,11 @@ public abstract class AbstractMessage extends AbstractMessageLite // should be prepared to accept them. int typeId = 0; - ByteString rawBytes = null; // If we encounter "message" before "typeId" - Message.Builder subBuilder = null; - FieldDescriptor field = null; + ByteString rawBytes = null; // If we encounter "message" before "typeId" + ExtensionRegistry.ExtensionInfo extension = null; + // Read bytes from input, if we get it's type first then parse it eagerly, + // otherwise we store the raw bytes in a local variable. while (true) { final int tag = input.readTag(); if (tag == 0) { @@ -516,75 +630,121 @@ public abstract class AbstractMessage extends AbstractMessageLite if (tag == WireFormat.MESSAGE_SET_TYPE_ID_TAG) { typeId = input.readUInt32(); - // Zero is not a valid type ID. if (typeId != 0) { - final ExtensionRegistry.ExtensionInfo extension; - // extensionRegistry may be either ExtensionRegistry or - // ExtensionRegistryLite. Since the type we are parsing is a full + // ExtensionRegistryLite. Since the type we are parsing is a full // message, only a full ExtensionRegistry could possibly contain - // extensions of it. Otherwise we will treat the registry as if it + // extensions of it. Otherwise we will treat the registry as if it // were empty. if (extensionRegistry instanceof ExtensionRegistry) { extension = ((ExtensionRegistry) extensionRegistry) .findExtensionByNumber(type, typeId); - } else { - extension = null; - } - - if (extension != null) { - field = extension.descriptor; - subBuilder = extension.defaultInstance.newBuilderForType(); - final Message originalMessage = (Message)builder.getField(field); - if (originalMessage != null) { - subBuilder.mergeFrom(originalMessage); - } - if (rawBytes != null) { - // We already encountered the message. Parse it now. - subBuilder.mergeFrom( - CodedInputStream.newInstance(rawBytes.newInput())); - rawBytes = null; - } - } else { - // Unknown extension number. If we already saw data, put it - // in rawBytes. - if (rawBytes != null) { - unknownFields.mergeField(typeId, - UnknownFieldSet.Field.newBuilder() - .addLengthDelimited(rawBytes) - .build()); - rawBytes = null; - } } } + } else if (tag == WireFormat.MESSAGE_SET_MESSAGE_TAG) { - if (typeId == 0) { - // We haven't seen a type ID yet, so we have to store the raw bytes - // for now. - rawBytes = input.readBytes(); - } else if (subBuilder == null) { - // We don't know how to parse this. Ignore it. - unknownFields.mergeField(typeId, - UnknownFieldSet.Field.newBuilder() - .addLengthDelimited(input.readBytes()) - .build()); - } else { - // We already know the type, so we can parse directly from the input - // with no copying. Hooray! - input.readMessage(subBuilder, extensionRegistry); + if (typeId != 0) { + if (extension != null && ExtensionRegistryLite.isEagerlyParseMessageSets()) { + // We already know the type, so we can parse directly from the + // input with no copying. Hooray! + eagerlyMergeMessageSetExtension( + input, extension, extensionRegistry, builder, extensions); + rawBytes = null; + continue; + } } - } else { - // Unknown tag. Skip it. + // We haven't seen a type ID yet or we want parse message lazily. + rawBytes = input.readBytes(); + + } else { // Unknown tag. Skip it. if (!input.skipField(tag)) { - break; // end of group + break; // End of group } } } - input.checkLastTagWas(WireFormat.MESSAGE_SET_ITEM_END_TAG); - if (subBuilder != null) { - builder.setField(field, subBuilder.build()); + // Process the raw bytes. + if (rawBytes != null && typeId != 0) { // Zero is not a valid type ID. + if (extension != null) { // We known the type + mergeMessageSetExtensionFromBytes( + rawBytes, extension, extensionRegistry, builder, extensions); + } else { // We don't know how to parse this. Ignore it. + if (rawBytes != null) { + unknownFields.mergeField(typeId, UnknownFieldSet.Field.newBuilder() + .addLengthDelimited(rawBytes).build()); + } + } + } + } + + private static void eagerlyMergeMessageSetExtension( + CodedInputStream input, + ExtensionRegistry.ExtensionInfo extension, + ExtensionRegistryLite extensionRegistry, + Message.Builder builder, + FieldSet<FieldDescriptor> extensions) throws IOException { + + FieldDescriptor field = extension.descriptor; + Message value = null; + if (hasOriginalMessage(builder, extensions, field)) { + Message originalMessage = + getOriginalMessage(builder, extensions, field); + Message.Builder subBuilder = originalMessage.toBuilder(); + input.readMessage(subBuilder, extensionRegistry); + value = subBuilder.buildPartial(); + } else { + value = input.readMessage(extension.defaultInstance.getParserForType(), + extensionRegistry); + } + + if (builder != null) { + builder.setField(field, value); + } else { + extensions.setField(field, value); + } + } + + private static void mergeMessageSetExtensionFromBytes( + ByteString rawBytes, + ExtensionRegistry.ExtensionInfo extension, + ExtensionRegistryLite extensionRegistry, + Message.Builder builder, + FieldSet<FieldDescriptor> extensions) throws IOException { + + FieldDescriptor field = extension.descriptor; + boolean hasOriginalValue = hasOriginalMessage(builder, extensions, field); + + if (hasOriginalValue || ExtensionRegistryLite.isEagerlyParseMessageSets()) { + // If the field already exists, we just parse the field. + Message value = null; + if (hasOriginalValue) { + Message originalMessage = + getOriginalMessage(builder, extensions, field); + Message.Builder subBuilder= originalMessage.toBuilder(); + subBuilder.mergeFrom(rawBytes, extensionRegistry); + value = subBuilder.buildPartial(); + } else { + value = extension.defaultInstance.getParserForType() + .parsePartialFrom(rawBytes, extensionRegistry); + } + setField(builder, extensions, field, value); + } else { + // Use LazyField to load MessageSet lazily. + LazyField lazyField = new LazyField( + extension.defaultInstance, extensionRegistry, rawBytes); + if (builder != null) { + // TODO(xiangl): it looks like this method can only be invoked by + // ExtendableBuilder, but I'm not sure. So I double check the type of + // builder here. It may be useless and need more investigation. + if (builder instanceof ExtendableBuilder) { + builder.setField(field, lazyField); + } else { + builder.setField(field, lazyField.getValue()); + } + } else { + extensions.setField(field, lazyField); + } } } @@ -596,6 +756,11 @@ public abstract class AbstractMessage extends AbstractMessageLite return (BuilderType) this; } + public Message.Builder getFieldBuilder(final FieldDescriptor field) { + throw new UnsupportedOperationException( + "getFieldBuilder() called on an unsupported message type."); + } + /** * Construct an UninitializedMessageException reporting missing fields in * the given message. @@ -609,14 +774,15 @@ public abstract class AbstractMessage extends AbstractMessageLite * Populates {@code this.missingFields} with the full "path" of each * missing required field in the given message. */ - private static List<String> findMissingFields(final Message message) { + private static List<String> findMissingFields( + final MessageOrBuilder message) { final List<String> results = new ArrayList<String>(); findMissingFields(message, "", results); return results; } /** Recursive helper implementing {@link #findMissingFields(Message)}. */ - private static void findMissingFields(final Message message, + private static void findMissingFields(final MessageOrBuilder message, final String prefix, final List<String> results) { for (final FieldDescriptor field : @@ -635,13 +801,13 @@ public abstract class AbstractMessage extends AbstractMessageLite if (field.isRepeated()) { int i = 0; for (final Object element : (List) value) { - findMissingFields((Message) element, + findMissingFields((MessageOrBuilder) element, subMessagePrefix(prefix, field, i++), results); } } else { if (message.hasField(field)) { - findMissingFields((Message) value, + findMissingFields((MessageOrBuilder) value, subMessagePrefix(prefix, field, -1), results); } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessageLite.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessageLite.java index 77b2737..9926f3d 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessageLite.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractMessageLite.java @@ -92,6 +92,14 @@ public abstract class AbstractMessageLite implements MessageLite { } /** + * Package private helper method for AbstractParser to create + * UninitializedMessageException. + */ + UninitializedMessageException newUninitializedMessageException() { + return new UninitializedMessageException(this); + } + + /** * A partial implementation of the {@link Message.Builder} interface which * implements as many methods of that interface as possible in terms of * other methods. @@ -307,10 +315,12 @@ public abstract class AbstractMessageLite implements MessageLite { */ protected static <T> void addAll(final Iterable<T> values, final Collection<? super T> list) { - for (final T value : values) { - if (value == null) { - throw new NullPointerException(); - } + if (values instanceof LazyStringList) { + // For StringOrByteStringLists, check the underlying elements to avoid + // forcing conversions of ByteStrings to Strings. + checkForNullValues(((LazyStringList) values).getUnderlyingElements()); + } else { + checkForNullValues(values); } if (values instanceof Collection) { final Collection<T> collection = (Collection<T>) values; @@ -321,5 +331,13 @@ public abstract class AbstractMessageLite implements MessageLite { } } } + + private static void checkForNullValues(final Iterable<?> values) { + for (final Object value : values) { + if (value == null) { + throw new NullPointerException(); + } + } + } } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractParser.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractParser.java new file mode 100644 index 0000000..9bd9d39 --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/AbstractParser.java @@ -0,0 +1,261 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import com.google.protobuf.AbstractMessageLite.Builder.LimitedInputStream; + +import java.io.IOException; +import java.io.InputStream; + +/** + * A partial implementation of the {@link Parser} interface which implements + * as many methods of that interface as possible in terms of other methods. + * + * Note: This class implements all the convenience methods in the + * {@link Parser} interface. See {@link Parser} for related javadocs. + * Subclasses need to implement + * {@link Parser#parsePartialFrom(CodedInputStream, ExtensionRegistryLite)} + * + * @author liujisi@google.com (Pherl Liu) + */ +public abstract class AbstractParser<MessageType extends MessageLite> + implements Parser<MessageType> { + /** + * Creates an UninitializedMessageException for MessageType. + */ + private UninitializedMessageException + newUninitializedMessageException(MessageType message) { + if (message instanceof AbstractMessageLite) { + return ((AbstractMessageLite) message).newUninitializedMessageException(); + } + return new UninitializedMessageException(message); + } + + /** + * Helper method to check if message is initialized. + * + * @throws InvalidProtocolBufferException if it is not initialized. + * @return The message to check. + */ + private MessageType checkMessageInitialized(MessageType message) + throws InvalidProtocolBufferException { + if (message != null && !message.isInitialized()) { + throw newUninitializedMessageException(message) + .asInvalidProtocolBufferException() + .setUnfinishedMessage(message); + } + return message; + } + + private static final ExtensionRegistryLite EMPTY_REGISTRY + = ExtensionRegistryLite.getEmptyRegistry(); + + public MessageType parsePartialFrom(CodedInputStream input) + throws InvalidProtocolBufferException { + return parsePartialFrom(input, EMPTY_REGISTRY); + } + + public MessageType parseFrom(CodedInputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return checkMessageInitialized( + parsePartialFrom(input, extensionRegistry)); + } + + public MessageType parseFrom(CodedInputStream input) + throws InvalidProtocolBufferException { + return parseFrom(input, EMPTY_REGISTRY); + } + + public MessageType parsePartialFrom(ByteString data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + MessageType message; + try { + CodedInputStream input = data.newCodedInput(); + message = parsePartialFrom(input, extensionRegistry); + try { + input.checkLastTagWas(0); + } catch (InvalidProtocolBufferException e) { + throw e.setUnfinishedMessage(message); + } + return message; + } catch (InvalidProtocolBufferException e) { + throw e; + } catch (IOException e) { + throw new RuntimeException( + "Reading from a ByteString threw an IOException (should " + + "never happen).", e); + } + } + + public MessageType parsePartialFrom(ByteString data) + throws InvalidProtocolBufferException { + return parsePartialFrom(data, EMPTY_REGISTRY); + } + + public MessageType parseFrom(ByteString data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return checkMessageInitialized(parsePartialFrom(data, extensionRegistry)); + } + + public MessageType parseFrom(ByteString data) + throws InvalidProtocolBufferException { + return parseFrom(data, EMPTY_REGISTRY); + } + + public MessageType parsePartialFrom(byte[] data, int off, int len, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + try { + CodedInputStream input = CodedInputStream.newInstance(data, off, len); + MessageType message = parsePartialFrom(input, extensionRegistry); + try { + input.checkLastTagWas(0); + } catch (InvalidProtocolBufferException e) { + throw e.setUnfinishedMessage(message); + } + return message; + } catch (InvalidProtocolBufferException e) { + throw e; + } catch (IOException e) { + throw new RuntimeException( + "Reading from a byte array threw an IOException (should " + + "never happen).", e); + } + } + + public MessageType parsePartialFrom(byte[] data, int off, int len) + throws InvalidProtocolBufferException { + return parsePartialFrom(data, off, len, EMPTY_REGISTRY); + } + + public MessageType parsePartialFrom(byte[] data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return parsePartialFrom(data, 0, data.length, extensionRegistry); + } + + public MessageType parsePartialFrom(byte[] data) + throws InvalidProtocolBufferException { + return parsePartialFrom(data, 0, data.length, EMPTY_REGISTRY); + } + + public MessageType parseFrom(byte[] data, int off, int len, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return checkMessageInitialized( + parsePartialFrom(data, off, len, extensionRegistry)); + } + + public MessageType parseFrom(byte[] data, int off, int len) + throws InvalidProtocolBufferException { + return parseFrom(data, off, len, EMPTY_REGISTRY); + } + + public MessageType parseFrom(byte[] data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return parseFrom(data, 0, data.length, extensionRegistry); + } + + public MessageType parseFrom(byte[] data) + throws InvalidProtocolBufferException { + return parseFrom(data, EMPTY_REGISTRY); + } + + public MessageType parsePartialFrom(InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + CodedInputStream codedInput = CodedInputStream.newInstance(input); + MessageType message = parsePartialFrom(codedInput, extensionRegistry); + try { + codedInput.checkLastTagWas(0); + } catch (InvalidProtocolBufferException e) { + throw e.setUnfinishedMessage(message); + } + return message; + } + + public MessageType parsePartialFrom(InputStream input) + throws InvalidProtocolBufferException { + return parsePartialFrom(input, EMPTY_REGISTRY); + } + + public MessageType parseFrom(InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return checkMessageInitialized( + parsePartialFrom(input, extensionRegistry)); + } + + public MessageType parseFrom(InputStream input) + throws InvalidProtocolBufferException { + return parseFrom(input, EMPTY_REGISTRY); + } + + public MessageType parsePartialDelimitedFrom( + InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + int size; + try { + int firstByte = input.read(); + if (firstByte == -1) { + return null; + } + size = CodedInputStream.readRawVarint32(firstByte, input); + } catch (IOException e) { + throw new InvalidProtocolBufferException(e.getMessage()); + } + InputStream limitedInput = new LimitedInputStream(input, size); + return parsePartialFrom(limitedInput, extensionRegistry); + } + + public MessageType parsePartialDelimitedFrom(InputStream input) + throws InvalidProtocolBufferException { + return parsePartialDelimitedFrom(input, EMPTY_REGISTRY); + } + + public MessageType parseDelimitedFrom( + InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + return checkMessageInitialized( + parsePartialDelimitedFrom(input, extensionRegistry)); + } + + public MessageType parseDelimitedFrom(InputStream input) + throws InvalidProtocolBufferException { + return parseDelimitedFrom(input, EMPTY_REGISTRY); + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/BoundedByteString.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/BoundedByteString.java new file mode 100644 index 0000000..cd4982c --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/BoundedByteString.java @@ -0,0 +1,163 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.util.NoSuchElementException; + +/** + * This class is used to represent the substring of a {@link ByteString} over a + * single byte array. In terms of the public API of {@link ByteString}, you end + * up here by calling {@link ByteString#copyFrom(byte[])} followed by {@link + * ByteString#substring(int, int)}. + * + * <p>This class contains most of the overhead involved in creating a substring + * from a {@link LiteralByteString}. The overhead involves some range-checking + * and two extra fields. + * + * @author carlanton@google.com (Carl Haverl) + */ +class BoundedByteString extends LiteralByteString { + + private final int bytesOffset; + private final int bytesLength; + + /** + * Creates a {@code BoundedByteString} backed by the sub-range of given array, + * without copying. + * + * @param bytes array to wrap + * @param offset index to first byte to use in bytes + * @param length number of bytes to use from bytes + * @throws IllegalArgumentException if {@code offset < 0}, {@code length < 0}, + * or if {@code offset + length > + * bytes.length}. + */ + BoundedByteString(byte[] bytes, int offset, int length) { + super(bytes); + if (offset < 0) { + throw new IllegalArgumentException("Offset too small: " + offset); + } + if (length < 0) { + throw new IllegalArgumentException("Length too small: " + offset); + } + if ((long) offset + length > bytes.length) { + throw new IllegalArgumentException( + "Offset+Length too large: " + offset + "+" + length); + } + + this.bytesOffset = offset; + this.bytesLength = length; + } + + /** + * Gets the byte at the given index. + * Throws {@link ArrayIndexOutOfBoundsException} + * for backwards-compatibility reasons although it would more properly be + * {@link IndexOutOfBoundsException}. + * + * @param index index of byte + * @return the value + * @throws ArrayIndexOutOfBoundsException {@code index} is < 0 or >= size + */ + @Override + public byte byteAt(int index) { + // We must check the index ourselves as we cannot rely on Java array index + // checking for substrings. + if (index < 0) { + throw new ArrayIndexOutOfBoundsException("Index too small: " + index); + } + if (index >= size()) { + throw new ArrayIndexOutOfBoundsException( + "Index too large: " + index + ", " + size()); + } + + return bytes[bytesOffset + index]; + } + + @Override + public int size() { + return bytesLength; + } + + @Override + protected int getOffsetIntoBytes() { + return bytesOffset; + } + + // ================================================================= + // ByteString -> byte[] + + @Override + protected void copyToInternal(byte[] target, int sourceOffset, + int targetOffset, int numberToCopy) { + System.arraycopy(bytes, getOffsetIntoBytes() + sourceOffset, target, + targetOffset, numberToCopy); + } + + // ================================================================= + // ByteIterator + + @Override + public ByteIterator iterator() { + return new BoundedByteIterator(); + } + + private class BoundedByteIterator implements ByteIterator { + + private int position; + private final int limit; + + private BoundedByteIterator() { + position = getOffsetIntoBytes(); + limit = position + size(); + } + + public boolean hasNext() { + return (position < limit); + } + + public Byte next() { + // Boxing calls Byte.valueOf(byte), which does not instantiate. + return nextByte(); + } + + public byte nextByte() { + if (position >= limit) { + throw new NoSuchElementException(); + } + return bytes[position++]; + } + + public void remove() { + throw new UnsupportedOperationException(); + } + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/ByteString.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/ByteString.java index 9135635..1b18169 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/ByteString.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/ByteString.java @@ -30,140 +30,413 @@ package com.google.protobuf; -import java.io.InputStream; -import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; -import java.io.FilterOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; import java.io.UnsupportedEncodingException; import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Iterator; import java.util.List; +import java.util.NoSuchElementException; /** - * Immutable array of bytes. + * Immutable sequence of bytes. Substring is supported by sharing the reference + * to the immutable underlying bytes, as with {@link String}. Concatenation is + * likewise supported without copying (long strings) by building a tree of + * pieces in {@link RopeByteString}. + * <p> + * Like {@link String}, the contents of a {@link ByteString} can never be + * observed to change, not even in the presence of a data race or incorrect + * API usage in the client code. * * @author crazybob@google.com Bob Lee * @author kenton@google.com Kenton Varda + * @author carlanton@google.com Carl Haverl + * @author martinrb@google.com Martin Buchholz */ -public final class ByteString { - private final byte[] bytes; +public abstract class ByteString implements Iterable<Byte> { - private ByteString(final byte[] bytes) { - this.bytes = bytes; - } + /** + * When two strings to be concatenated have a combined length shorter than + * this, we just copy their bytes on {@link #concat(ByteString)}. + * The trade-off is copy size versus the overhead of creating tree nodes + * in {@link RopeByteString}. + */ + static final int CONCATENATE_BY_COPY_SIZE = 128; + + /** + * When copying an InputStream into a ByteString with .readFrom(), + * the chunks in the underlying rope start at 256 bytes, but double + * each iteration up to 8192 bytes. + */ + static final int MIN_READ_FROM_CHUNK_SIZE = 0x100; // 256b + static final int MAX_READ_FROM_CHUNK_SIZE = 0x2000; // 8k /** - * Gets the byte at the given index. + * Empty {@code ByteString}. + */ + public static final ByteString EMPTY = new LiteralByteString(new byte[0]); + + // This constructor is here to prevent subclassing outside of this package, + ByteString() {} + + /** + * Gets the byte at the given index. This method should be used only for + * random access to individual bytes. To access bytes sequentially, use the + * {@link ByteIterator} returned by {@link #iterator()}, and call {@link + * #substring(int, int)} first if necessary. * + * @param index index of byte + * @return the value * @throws ArrayIndexOutOfBoundsException {@code index} is < 0 or >= size */ - public byte byteAt(final int index) { - return bytes[index]; + public abstract byte byteAt(int index); + + /** + * Return a {@link ByteString.ByteIterator} over the bytes in the ByteString. + * To avoid auto-boxing, you may get the iterator manually and call + * {@link ByteIterator#nextByte()}. + * + * @return the iterator + */ + public abstract ByteIterator iterator(); + + /** + * This interface extends {@code Iterator<Byte>}, so that we can return an + * unboxed {@code byte}. + */ + public interface ByteIterator extends Iterator<Byte> { + /** + * An alternative to {@link Iterator#next()} that returns an + * unboxed primitive {@code byte}. + * + * @return the next {@code byte} in the iteration + * @throws NoSuchElementException if the iteration has no more elements + */ + byte nextByte(); } /** * Gets the number of bytes. + * + * @return size in bytes */ - public int size() { - return bytes.length; - } + public abstract int size(); /** * Returns {@code true} if the size is {@code 0}, {@code false} otherwise. + * + * @return true if this is zero bytes long */ public boolean isEmpty() { - return bytes.length == 0; + return size() == 0; } // ================================================================= - // byte[] -> ByteString + // ByteString -> substring /** - * Empty ByteString. + * Return the substring from {@code beginIndex}, inclusive, to the end of the + * string. + * + * @param beginIndex start at this index + * @return substring sharing underlying data + * @throws IndexOutOfBoundsException if {@code beginIndex < 0} or + * {@code beginIndex > size()}. */ - public static final ByteString EMPTY = new ByteString(new byte[0]); + public ByteString substring(int beginIndex) { + return substring(beginIndex, size()); + } + + /** + * Return the substring from {@code beginIndex}, inclusive, to {@code + * endIndex}, exclusive. + * + * @param beginIndex start at this index + * @param endIndex the last character is the one before this index + * @return substring sharing underlying data + * @throws IndexOutOfBoundsException if {@code beginIndex < 0}, + * {@code endIndex > size()}, or {@code beginIndex > endIndex}. + */ + public abstract ByteString substring(int beginIndex, int endIndex); + + /** + * Tests if this bytestring starts with the specified prefix. + * Similar to {@link String#startsWith(String)} + * + * @param prefix the prefix. + * @return <code>true</code> if the byte sequence represented by the + * argument is a prefix of the byte sequence represented by + * this string; <code>false</code> otherwise. + */ + public boolean startsWith(ByteString prefix) { + return size() >= prefix.size() && + substring(0, prefix.size()).equals(prefix); + } + + // ================================================================= + // byte[] -> ByteString /** * Copies the given bytes into a {@code ByteString}. + * + * @param bytes source array + * @param offset offset in source array + * @param size number of bytes to copy + * @return new {@code ByteString} */ - public static ByteString copyFrom(final byte[] bytes, final int offset, - final int size) { - final byte[] copy = new byte[size]; + public static ByteString copyFrom(byte[] bytes, int offset, int size) { + byte[] copy = new byte[size]; System.arraycopy(bytes, offset, copy, 0, size); - return new ByteString(copy); + return new LiteralByteString(copy); } /** * Copies the given bytes into a {@code ByteString}. + * + * @param bytes to copy + * @return new {@code ByteString} */ - public static ByteString copyFrom(final byte[] bytes) { + public static ByteString copyFrom(byte[] bytes) { return copyFrom(bytes, 0, bytes.length); } /** - * Copies {@code size} bytes from a {@code java.nio.ByteBuffer} into + * Copies the next {@code size} bytes from a {@code java.nio.ByteBuffer} into * a {@code ByteString}. + * + * @param bytes source buffer + * @param size number of bytes to copy + * @return new {@code ByteString} */ - public static ByteString copyFrom(final ByteBuffer bytes, final int size) { - final byte[] copy = new byte[size]; + public static ByteString copyFrom(ByteBuffer bytes, int size) { + byte[] copy = new byte[size]; bytes.get(copy); - return new ByteString(copy); + return new LiteralByteString(copy); } /** * Copies the remaining bytes from a {@code java.nio.ByteBuffer} into * a {@code ByteString}. + * + * @param bytes sourceBuffer + * @return new {@code ByteString} */ - public static ByteString copyFrom(final ByteBuffer bytes) { + public static ByteString copyFrom(ByteBuffer bytes) { return copyFrom(bytes, bytes.remaining()); } /** * Encodes {@code text} into a sequence of bytes using the named charset * and returns the result as a {@code ByteString}. + * + * @param text source string + * @param charsetName encoding to use + * @return new {@code ByteString} + * @throws UnsupportedEncodingException if the encoding isn't found */ - public static ByteString copyFrom(final String text, final String charsetName) + public static ByteString copyFrom(String text, String charsetName) throws UnsupportedEncodingException { - return new ByteString(text.getBytes(charsetName)); + return new LiteralByteString(text.getBytes(charsetName)); } /** * Encodes {@code text} into a sequence of UTF-8 bytes and returns the * result as a {@code ByteString}. + * + * @param text source string + * @return new {@code ByteString} */ - public static ByteString copyFromUtf8(final String text) { + public static ByteString copyFromUtf8(String text) { try { - return new ByteString(text.getBytes("UTF-8")); + return new LiteralByteString(text.getBytes("UTF-8")); } catch (UnsupportedEncodingException e) { throw new RuntimeException("UTF-8 not supported?", e); } } + // ================================================================= + // InputStream -> ByteString + /** - * Concatenates all byte strings in the list and returns the result. + * Completely reads the given stream's bytes into a + * {@code ByteString}, blocking if necessary until all bytes are + * read through to the end of the stream. + * + * <b>Performance notes:</b> The returned {@code ByteString} is an + * immutable tree of byte arrays ("chunks") of the stream data. The + * first chunk is small, with subsequent chunks each being double + * the size, up to 8K. If the caller knows the precise length of + * the stream and wishes to avoid all unnecessary copies and + * allocations, consider using the two-argument version of this + * method, below. + * + * @param streamToDrain The source stream, which is read completely + * but not closed. + * @return A new {@code ByteString} which is made up of chunks of + * various sizes, depending on the behavior of the underlying + * stream. + * @throws IOException IOException is thrown if there is a problem + * reading the underlying stream. + */ + public static ByteString readFrom(InputStream streamToDrain) + throws IOException { + return readFrom( + streamToDrain, MIN_READ_FROM_CHUNK_SIZE, MAX_READ_FROM_CHUNK_SIZE); + } + + /** + * Completely reads the given stream's bytes into a + * {@code ByteString}, blocking if necessary until all bytes are + * read through to the end of the stream. + * + * <b>Performance notes:</b> The returned {@code ByteString} is an + * immutable tree of byte arrays ("chunks") of the stream data. The + * chunkSize parameter sets the size of these byte arrays. In + * particular, if the chunkSize is precisely the same as the length + * of the stream, unnecessary allocations and copies will be + * avoided. Otherwise, the chunks will be of the given size, except + * for the last chunk, which will be resized (via a reallocation and + * copy) to contain the remainder of the stream. + * + * @param streamToDrain The source stream, which is read completely + * but not closed. + * @param chunkSize The size of the chunks in which to read the + * stream. + * @return A new {@code ByteString} which is made up of chunks of + * the given size. + * @throws IOException IOException is thrown if there is a problem + * reading the underlying stream. + */ + public static ByteString readFrom(InputStream streamToDrain, int chunkSize) + throws IOException { + return readFrom(streamToDrain, chunkSize, chunkSize); + } + + // Helper method that takes the chunk size range as a parameter. + public static ByteString readFrom(InputStream streamToDrain, int minChunkSize, + int maxChunkSize) throws IOException { + Collection<ByteString> results = new ArrayList<ByteString>(); + + // copy the inbound bytes into a list of chunks; the chunk size + // grows exponentially to support both short and long streams. + int chunkSize = minChunkSize; + while (true) { + ByteString chunk = readChunk(streamToDrain, chunkSize); + if (chunk == null) { + break; + } + results.add(chunk); + chunkSize = Math.min(chunkSize * 2, maxChunkSize); + } + + return ByteString.copyFrom(results); + } + + /** + * Blocks until a chunk of the given size can be made from the + * stream, or EOF is reached. Calls read() repeatedly in case the + * given stream implementation doesn't completely fill the given + * buffer in one read() call. + * + * @return A chunk of the desired size, or else a chunk as large as + * was available when end of stream was reached. Returns null if the + * given stream had no more data in it. + */ + private static ByteString readChunk(InputStream in, final int chunkSize) + throws IOException { + final byte[] buf = new byte[chunkSize]; + int bytesRead = 0; + while (bytesRead < chunkSize) { + final int count = in.read(buf, bytesRead, chunkSize - bytesRead); + if (count == -1) { + break; + } + bytesRead += count; + } + + if (bytesRead == 0) { + return null; + } else { + return ByteString.copyFrom(buf, 0, bytesRead); + } + } + + // ================================================================= + // Multiple ByteStrings -> One ByteString + + /** + * Concatenate the given {@code ByteString} to this one. Short concatenations, + * of total size smaller than {@link ByteString#CONCATENATE_BY_COPY_SIZE}, are + * produced by copying the underlying bytes (as per Rope.java, <a + * href="http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol25/issue12/spe986.pdf"> + * BAP95 </a>. In general, the concatenate involves no copying. + * + * @param other string to concatenate + * @return a new {@code ByteString} instance + */ + public ByteString concat(ByteString other) { + int thisSize = size(); + int otherSize = other.size(); + if ((long) thisSize + otherSize >= Integer.MAX_VALUE) { + throw new IllegalArgumentException("ByteString would be too long: " + + thisSize + "+" + otherSize); + } + + return RopeByteString.concatenate(this, other); + } + + /** + * Concatenates all byte strings in the iterable and returns the result. + * This is designed to run in O(list size), not O(total bytes). * * <p>The returned {@code ByteString} is not necessarily a unique object. * If the list is empty, the returned object is the singleton empty * {@code ByteString}. If the list has only one element, that * {@code ByteString} will be returned without copying. + * + * @param byteStrings strings to be concatenated + * @return new {@code ByteString} */ - public static ByteString copyFrom(List<ByteString> list) { - if (list.size() == 0) { - return EMPTY; - } else if (list.size() == 1) { - return list.get(0); + public static ByteString copyFrom(Iterable<ByteString> byteStrings) { + Collection<ByteString> collection; + if (!(byteStrings instanceof Collection)) { + collection = new ArrayList<ByteString>(); + for (ByteString byteString : byteStrings) { + collection.add(byteString); + } + } else { + collection = (Collection<ByteString>) byteStrings; } - - int size = 0; - for (ByteString str : list) { - size += str.size(); + ByteString result; + if (collection.isEmpty()) { + result = EMPTY; + } else { + result = balancedConcat(collection.iterator(), collection.size()); } - byte[] bytes = new byte[size]; - int pos = 0; - for (ByteString str : list) { - System.arraycopy(str.bytes, 0, bytes, pos, str.size()); - pos += str.size(); + return result; + } + + // Internal function used by copyFrom(Iterable<ByteString>). + // Create a balanced concatenation of the next "length" elements from the + // iterable. + private static ByteString balancedConcat(Iterator<ByteString> iterator, + int length) { + assert length >= 1; + ByteString result; + if (length == 1) { + result = iterator.next(); + } else { + int halfLength = length >>> 1; + ByteString left = balancedConcat(iterator, halfLength); + ByteString right = balancedConcat(iterator, length - halfLength); + result = left.concat(right); } - return new ByteString(bytes); + return result; } // ================================================================= @@ -174,206 +447,446 @@ public final class ByteString { * * @param target buffer to copy into * @param offset in the target buffer + * @throws IndexOutOfBoundsException if the offset is negative or too large */ - public void copyTo(final byte[] target, final int offset) { - System.arraycopy(bytes, 0, target, offset, bytes.length); + public void copyTo(byte[] target, int offset) { + copyTo(target, 0, offset, size()); } /** * Copies bytes into a buffer. * - * @param target buffer to copy into + * @param target buffer to copy into * @param sourceOffset offset within these bytes * @param targetOffset offset within the target buffer - * @param size number of bytes to copy + * @param numberToCopy number of bytes to copy + * @throws IndexOutOfBoundsException if an offset or size is negative or too + * large */ - public void copyTo(final byte[] target, final int sourceOffset, - final int targetOffset, - final int size) { - System.arraycopy(bytes, sourceOffset, target, targetOffset, size); + public void copyTo(byte[] target, int sourceOffset, int targetOffset, + int numberToCopy) { + if (sourceOffset < 0) { + throw new IndexOutOfBoundsException("Source offset < 0: " + sourceOffset); + } + if (targetOffset < 0) { + throw new IndexOutOfBoundsException("Target offset < 0: " + targetOffset); + } + if (numberToCopy < 0) { + throw new IndexOutOfBoundsException("Length < 0: " + numberToCopy); + } + if (sourceOffset + numberToCopy > size()) { + throw new IndexOutOfBoundsException( + "Source end offset < 0: " + (sourceOffset + numberToCopy)); + } + if (targetOffset + numberToCopy > target.length) { + throw new IndexOutOfBoundsException( + "Target end offset < 0: " + (targetOffset + numberToCopy)); + } + if (numberToCopy > 0) { + copyToInternal(target, sourceOffset, targetOffset, numberToCopy); + } } /** + * Internal (package private) implementation of + * @link{#copyTo(byte[],int,int,int}. + * It assumes that all error checking has already been performed and that + * @code{numberToCopy > 0}. + */ + protected abstract void copyToInternal(byte[] target, int sourceOffset, + int targetOffset, int numberToCopy); + + /** * Copies bytes into a ByteBuffer. * * @param target ByteBuffer to copy into. - * @throws ReadOnlyBufferException if the {@code target} is read-only - * @throws BufferOverflowException if the {@code target}'s remaining() - * space is not large enough to hold the data. + * @throws java.nio.ReadOnlyBufferException if the {@code target} is read-only + * @throws java.nio.BufferOverflowException if the {@code target}'s + * remaining() space is not large enough to hold the data. */ - public void copyTo(ByteBuffer target) { - target.put(bytes, 0, bytes.length); - } + public abstract void copyTo(ByteBuffer target); /** * Copies bytes to a {@code byte[]}. + * + * @return copied bytes */ public byte[] toByteArray() { - final int size = bytes.length; - final byte[] copy = new byte[size]; - System.arraycopy(bytes, 0, copy, 0, size); - return copy; + int size = size(); + byte[] result = new byte[size]; + copyToInternal(result, 0, 0, size); + return result; } /** - * Constructs a new read-only {@code java.nio.ByteBuffer} with the - * same backing byte array. + * Writes the complete contents of this byte string to + * the specified output stream argument. + * + * @param out the output stream to which to write the data. + * @throws IOException if an I/O error occurs. */ - public ByteBuffer asReadOnlyByteBuffer() { - final ByteBuffer byteBuffer = ByteBuffer.wrap(bytes); - return byteBuffer.asReadOnlyBuffer(); - } + public abstract void writeTo(OutputStream out) throws IOException; + + /** + * Constructs a read-only {@code java.nio.ByteBuffer} whose content + * is equal to the contents of this byte string. + * The result uses the same backing array as the byte string, if possible. + * + * @return wrapped bytes + */ + public abstract ByteBuffer asReadOnlyByteBuffer(); + + /** + * Constructs a list of read-only {@code java.nio.ByteBuffer} objects + * such that the concatenation of their contents is equal to the contents + * of this byte string. The result uses the same backing arrays as the + * byte string. + * <p> + * By returning a list, implementations of this method may be able to avoid + * copying even when there are multiple backing arrays. + * + * @return a list of wrapped bytes + */ + public abstract List<ByteBuffer> asReadOnlyByteBufferList(); /** * Constructs a new {@code String} by decoding the bytes using the * specified charset. + * + * @param charsetName encode using this charset + * @return new string + * @throws UnsupportedEncodingException if charset isn't recognized */ - public String toString(final String charsetName) - throws UnsupportedEncodingException { - return new String(bytes, charsetName); - } + public abstract String toString(String charsetName) + throws UnsupportedEncodingException; + + // ================================================================= + // UTF-8 decoding /** * Constructs a new {@code String} by decoding the bytes as UTF-8. + * + * @return new string using UTF-8 encoding */ public String toStringUtf8() { try { - return new String(bytes, "UTF-8"); + return toString("UTF-8"); } catch (UnsupportedEncodingException e) { throw new RuntimeException("UTF-8 not supported?", e); } } + /** + * Tells whether this {@code ByteString} represents a well-formed UTF-8 + * byte sequence, such that the original bytes can be converted to a + * String object and then round tripped back to bytes without loss. + * + * <p>More precisely, returns {@code true} whenever: <pre> {@code + * Arrays.equals(byteString.toByteArray(), + * new String(byteString.toByteArray(), "UTF-8").getBytes("UTF-8")) + * }</pre> + * + * <p>This method returns {@code false} for "overlong" byte sequences, + * as well as for 3-byte sequences that would map to a surrogate + * character, in accordance with the restricted definition of UTF-8 + * introduced in Unicode 3.1. Note that the UTF-8 decoder included in + * Oracle's JDK has been modified to also reject "overlong" byte + * sequences, but (as of 2011) still accepts 3-byte surrogate + * character byte sequences. + * + * <p>See the Unicode Standard,</br> + * Table 3-6. <em>UTF-8 Bit Distribution</em>,</br> + * Table 3-7. <em>Well Formed UTF-8 Byte Sequences</em>. + * + * @return whether the bytes in this {@code ByteString} are a + * well-formed UTF-8 byte sequence + */ + public abstract boolean isValidUtf8(); + + /** + * Tells whether the given byte sequence is a well-formed, malformed, or + * incomplete UTF-8 byte sequence. This method accepts and returns a partial + * state result, allowing the bytes for a complete UTF-8 byte sequence to be + * composed from multiple {@code ByteString} segments. + * + * @param state either {@code 0} (if this is the initial decoding operation) + * or the value returned from a call to a partial decoding method for the + * previous bytes + * @param offset offset of the first byte to check + * @param length number of bytes to check + * + * @return {@code -1} if the partial byte sequence is definitely malformed, + * {@code 0} if it is well-formed (no additional input needed), or, if the + * byte sequence is "incomplete", i.e. apparently terminated in the middle of + * a character, an opaque integer "state" value containing enough information + * to decode the character when passed to a subsequent invocation of a + * partial decoding method. + */ + protected abstract int partialIsValidUtf8(int state, int offset, int length); + // ================================================================= // equals() and hashCode() @Override - public boolean equals(final Object o) { - if (o == this) { - return true; - } - - if (!(o instanceof ByteString)) { - return false; - } - - final ByteString other = (ByteString) o; - final int size = bytes.length; - if (size != other.bytes.length) { - return false; - } - - final byte[] thisBytes = bytes; - final byte[] otherBytes = other.bytes; - for (int i = 0; i < size; i++) { - if (thisBytes[i] != otherBytes[i]) { - return false; - } - } - - return true; - } - - private volatile int hash = 0; + public abstract boolean equals(Object o); + /** + * Return a non-zero hashCode depending only on the sequence of bytes + * in this ByteString. + * + * @return hashCode value for this object + */ @Override - public int hashCode() { - int h = hash; - - if (h == 0) { - final byte[] thisBytes = bytes; - final int size = bytes.length; - - h = size; - for (int i = 0; i < size; i++) { - h = h * 31 + thisBytes[i]; - } - if (h == 0) { - h = 1; - } - - hash = h; - } - - return h; - } + public abstract int hashCode(); // ================================================================= // Input stream /** * Creates an {@code InputStream} which can be used to read the bytes. + * <p> + * The {@link InputStream} returned by this method is guaranteed to be + * completely non-blocking. The method {@link InputStream#available()} + * returns the number of bytes remaining in the stream. The methods + * {@link InputStream#read(byte[]), {@link InputStream#read(byte[],int,int)} + * and {@link InputStream#skip(long)} will read/skip as many bytes as are + * available. + * <p> + * The methods in the returned {@link InputStream} might <b>not</b> be + * thread safe. + * + * @return an input stream that returns the bytes of this byte string. */ - public InputStream newInput() { - return new ByteArrayInputStream(bytes); - } + public abstract InputStream newInput(); /** * Creates a {@link CodedInputStream} which can be used to read the bytes. - * Using this is more efficient than creating a {@link CodedInputStream} - * wrapping the result of {@link #newInput()}. + * Using this is often more efficient than creating a {@link CodedInputStream} + * that wraps the result of {@link #newInput()}. + * + * @return stream based on wrapped data */ - public CodedInputStream newCodedInput() { - // We trust CodedInputStream not to modify the bytes, or to give anyone - // else access to them. - return CodedInputStream.newInstance(bytes); - } + public abstract CodedInputStream newCodedInput(); // ================================================================= // Output stream /** - * Creates a new {@link Output} with the given initial capacity. + * Creates a new {@link Output} with the given initial capacity. Call {@link + * Output#toByteString()} to create the {@code ByteString} instance. + * <p> + * A {@link ByteString.Output} offers the same functionality as a + * {@link ByteArrayOutputStream}, except that it returns a {@link ByteString} + * rather than a {@code byte} array. + * + * @param initialCapacity estimate of number of bytes to be written + * @return {@code OutputStream} for building a {@code ByteString} */ - public static Output newOutput(final int initialCapacity) { - return new Output(new ByteArrayOutputStream(initialCapacity)); + public static Output newOutput(int initialCapacity) { + return new Output(initialCapacity); } /** - * Creates a new {@link Output}. + * Creates a new {@link Output}. Call {@link Output#toByteString()} to create + * the {@code ByteString} instance. + * <p> + * A {@link ByteString.Output} offers the same functionality as a + * {@link ByteArrayOutputStream}, except that it returns a {@link ByteString} + * rather than a {@code byte array}. + * + * @return {@code OutputStream} for building a {@code ByteString} */ public static Output newOutput() { - return newOutput(32); + return new Output(CONCATENATE_BY_COPY_SIZE); } /** * Outputs to a {@code ByteString} instance. Call {@link #toByteString()} to * create the {@code ByteString} instance. */ - public static final class Output extends FilterOutputStream { - private final ByteArrayOutputStream bout; + public static final class Output extends OutputStream { + // Implementation note. + // The public methods of this class must be synchronized. ByteStrings + // are guaranteed to be immutable. Without some sort of locking, it could + // be possible for one thread to call toByteSring(), while another thread + // is still modifying the underlying byte array. + + private static final byte[] EMPTY_BYTE_ARRAY = new byte[0]; + // argument passed by user, indicating initial capacity. + private final int initialCapacity; + // ByteStrings to be concatenated to create the result + private final ArrayList<ByteString> flushedBuffers; + // Total number of bytes in the ByteStrings of flushedBuffers + private int flushedBuffersTotalBytes; + // Current buffer to which we are writing + private byte[] buffer; + // Location in buffer[] to which we write the next byte. + private int bufferPos; /** - * Constructs a new output with the given initial capacity. + * Creates a new ByteString output stream with the specified + * initial capacity. + * + * @param initialCapacity the initial capacity of the output stream. */ - private Output(final ByteArrayOutputStream bout) { - super(bout); - this.bout = bout; + Output(int initialCapacity) { + if (initialCapacity < 0) { + throw new IllegalArgumentException("Buffer size < 0"); + } + this.initialCapacity = initialCapacity; + this.flushedBuffers = new ArrayList<ByteString>(); + this.buffer = new byte[initialCapacity]; + } + + @Override + public synchronized void write(int b) { + if (bufferPos == buffer.length) { + flushFullBuffer(1); + } + buffer[bufferPos++] = (byte)b; + } + + @Override + public synchronized void write(byte[] b, int offset, int length) { + if (length <= buffer.length - bufferPos) { + // The bytes can fit into the current buffer. + System.arraycopy(b, offset, buffer, bufferPos, length); + bufferPos += length; + } else { + // Use up the current buffer + int copySize = buffer.length - bufferPos; + System.arraycopy(b, offset, buffer, bufferPos, copySize); + offset += copySize; + length -= copySize; + // Flush the buffer, and get a new buffer at least big enough to cover + // what we still need to output + flushFullBuffer(length); + System.arraycopy(b, offset, buffer, 0 /* count */, length); + bufferPos = length; + } + } + + /** + * Creates a byte string. Its size is the current size of this output + * stream and its output has been copied to it. + * + * @return the current contents of this output stream, as a byte string. + */ + public synchronized ByteString toByteString() { + flushLastBuffer(); + return ByteString.copyFrom(flushedBuffers); + } + + /** + * Writes the complete contents of this byte array output stream to + * the specified output stream argument. + * + * @param out the output stream to which to write the data. + * @throws IOException if an I/O error occurs. + */ + public void writeTo(OutputStream out) throws IOException { + ByteString[] cachedFlushBuffers; + byte[] cachedBuffer; + int cachedBufferPos; + synchronized (this) { + // Copy the information we need into local variables so as to hold + // the lock for as short a time as possible. + cachedFlushBuffers = + flushedBuffers.toArray(new ByteString[flushedBuffers.size()]); + cachedBuffer = buffer; + cachedBufferPos = bufferPos; + } + for (ByteString byteString : cachedFlushBuffers) { + byteString.writeTo(out); + } + + out.write(Arrays.copyOf(cachedBuffer, cachedBufferPos)); + } + + /** + * Returns the current size of the output stream. + * + * @return the current size of the output stream + */ + public synchronized int size() { + return flushedBuffersTotalBytes + bufferPos; + } + + /** + * Resets this stream, so that all currently accumulated output in the + * output stream is discarded. The output stream can be used again, + * reusing the already allocated buffer space. + */ + public synchronized void reset() { + flushedBuffers.clear(); + flushedBuffersTotalBytes = 0; + bufferPos = 0; + } + + @Override + public String toString() { + return String.format("<ByteString.Output@%s size=%d>", + Integer.toHexString(System.identityHashCode(this)), size()); } /** - * Creates a {@code ByteString} instance from this {@code Output}. + * Internal function used by writers. The current buffer is full, and the + * writer needs a new buffer whose size is at least the specified minimum + * size. */ - public ByteString toByteString() { - final byte[] byteArray = bout.toByteArray(); - return new ByteString(byteArray); + private void flushFullBuffer(int minSize) { + flushedBuffers.add(new LiteralByteString(buffer)); + flushedBuffersTotalBytes += buffer.length; + // We want to increase our total capacity by 50%, but as a minimum, + // the new buffer should also at least be >= minSize and + // >= initial Capacity. + int newSize = Math.max(initialCapacity, + Math.max(minSize, flushedBuffersTotalBytes >>> 1)); + buffer = new byte[newSize]; + bufferPos = 0; + } + + /** + * Internal function used by {@link #toByteString()}. The current buffer may + * or may not be full, but it needs to be flushed. + */ + private void flushLastBuffer() { + if (bufferPos < buffer.length) { + if (bufferPos > 0) { + byte[] bufferCopy = Arrays.copyOf(buffer, bufferPos); + flushedBuffers.add(new LiteralByteString(bufferCopy)); + } + // We reuse this buffer for further writes. + } else { + // Buffer is completely full. Huzzah. + flushedBuffers.add(new LiteralByteString(buffer)); + // 99% of the time, we're not going to use this OutputStream again. + // We set buffer to an empty byte stream so that we're handling this + // case without wasting space. In the rare case that more writes + // *do* occur, this empty buffer will be flushed and an appropriately + // sized new buffer will be created. + buffer = EMPTY_BYTE_ARRAY; + } + flushedBuffersTotalBytes += bufferPos; + bufferPos = 0; } } /** - * Constructs a new ByteString builder, which allows you to efficiently - * construct a {@code ByteString} by writing to a {@link CodedOutputStream}. - * Using this is much more efficient than calling {@code newOutput()} and - * wrapping that in a {@code CodedOutputStream}. + * Constructs a new {@code ByteString} builder, which allows you to + * efficiently construct a {@code ByteString} by writing to a {@link + * CodedOutputStream}. Using this is much more efficient than calling {@code + * newOutput()} and wrapping that in a {@code CodedOutputStream}. * * <p>This is package-private because it's a somewhat confusing interface. * Users can call {@link Message#toByteString()} instead of calling this * directly. * - * @param size The target byte size of the {@code ByteString}. You must - * write exactly this many bytes before building the result. + * @param size The target byte size of the {@code ByteString}. You must write + * exactly this many bytes before building the result. + * @return the builder */ - static CodedBuilder newCodedBuilder(final int size) { + static CodedBuilder newCodedBuilder(int size) { return new CodedBuilder(size); } @@ -382,7 +895,7 @@ public final class ByteString { private final CodedOutputStream output; private final byte[] buffer; - private CodedBuilder(final int size) { + private CodedBuilder(int size) { buffer = new byte[size]; output = CodedOutputStream.newInstance(buffer); } @@ -393,11 +906,57 @@ public final class ByteString { // We can be confident that the CodedOutputStream will not modify the // underlying bytes anymore because it already wrote all of them. So, // no need to make a copy. - return new ByteString(buffer); + return new LiteralByteString(buffer); } public CodedOutputStream getCodedOutput() { return output; } } + + // ================================================================= + // Methods {@link RopeByteString} needs on instances, which aren't part of the + // public API. + + /** + * Return the depth of the tree representing this {@code ByteString}, if any, + * whose root is this node. If this is a leaf node, return 0. + * + * @return tree depth or zero + */ + protected abstract int getTreeDepth(); + + /** + * Return {@code true} if this ByteString is literal (a leaf node) or a + * flat-enough tree in the sense of {@link RopeByteString}. + * + * @return true if the tree is flat enough + */ + protected abstract boolean isBalanced(); + + /** + * Return the cached hash code if available. + * + * @return value of cached hash code or 0 if not computed yet + */ + protected abstract int peekCachedHashCode(); + + /** + * Compute the hash across the value bytes starting with the given hash, and + * return the result. This is used to compute the hash across strings + * represented as a set of pieces by allowing the hash computation to be + * continued from piece to piece. + * + * @param h starting hash value + * @param offset offset into this value to start looking at data values + * @param length number of data values to include in the hash computation + * @return ending hash value + */ + protected abstract int partialHash(int h, int offset, int length); + + @Override + public String toString() { + return String.format("<ByteString@%s size=%d>", + Integer.toHexString(System.identityHashCode(this)), size()); + } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedInputStream.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedInputStream.java index b3e0855..33417a7 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedInputStream.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedInputStream.java @@ -243,6 +243,23 @@ public final class CodedInputStream { --recursionDepth; } + /** Read a {@code group} field value from the stream. */ + public <T extends MessageLite> T readGroup( + final int fieldNumber, + final Parser<T> parser, + final ExtensionRegistryLite extensionRegistry) + throws IOException { + if (recursionDepth >= recursionLimit) { + throw InvalidProtocolBufferException.recursionLimitExceeded(); + } + ++recursionDepth; + T result = parser.parsePartialFrom(this, extensionRegistry); + checkLastTagWas( + WireFormat.makeTag(fieldNumber, WireFormat.WIRETYPE_END_GROUP)); + --recursionDepth; + return result; + } + /** * Reads a {@code group} field value from the stream and merges it into the * given {@link UnknownFieldSet}. @@ -278,6 +295,24 @@ public final class CodedInputStream { popLimit(oldLimit); } + /** Read an embedded message field value from the stream. */ + public <T extends MessageLite> T readMessage( + final Parser<T> parser, + final ExtensionRegistryLite extensionRegistry) + throws IOException { + int length = readRawVarint32(); + if (recursionDepth >= recursionLimit) { + throw InvalidProtocolBufferException.recursionLimitExceeded(); + } + final int oldLimit = pushLimit(length); + ++recursionDepth; + T result = parser.parsePartialFrom(this, extensionRegistry); + checkLastTagWas(0); + --recursionDepth; + popLimit(oldLimit); + return result; + } + /** Read a {@code bytes} field value from the stream. */ public ByteString readBytes() throws IOException { final int size = readRawVarint32(); @@ -601,7 +636,7 @@ public final class CodedInputStream { * refreshing its buffer. If you need to prevent reading past a certain * point in the underlying {@code InputStream} (e.g. because you expect it to * contain more data after the end of the message which you need to handle - * differently) then you must place a wrapper around you {@code InputStream} + * differently) then you must place a wrapper around your {@code InputStream} * which limits the amount of data that can be read from it. * * @return the old limit. @@ -676,7 +711,7 @@ public final class CodedInputStream { /** * Called with {@code this.buffer} is empty to read more bytes from the - * input. If {@code mustSucceed} is true, refillBuffer() gurantees that + * input. If {@code mustSucceed} is true, refillBuffer() guarantees that * either there will be at least one byte in the buffer when it returns * or it will throw an exception. If {@code mustSucceed} is false, * refillBuffer() returns false if no more bytes were available. @@ -879,7 +914,7 @@ public final class CodedInputStream { refillBuffer(true); } - bufferPos = size - pos; + bufferPos = size - pos; } } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedOutputStream.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedOutputStream.java index ac5f2d3..ca24638 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedOutputStream.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/CodedOutputStream.java @@ -30,10 +30,10 @@ package com.google.protobuf; -import java.io.OutputStream; import java.io.IOException; -import java.io.UnsupportedEncodingException; import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; /** * Encodes and writes protocol message fields. @@ -540,6 +540,15 @@ public final class CodedOutputStream { } /** + * Compute the number of bytes that would be needed to encode an + * embedded message in lazy field, including tag. + */ + public static int computeLazyFieldSize(final int fieldNumber, + final LazyField value) { + return computeTagSize(fieldNumber) + computeLazyFieldSizeNoTag(value); + } + + /** * Compute the number of bytes that would be needed to encode a * {@code uint32} field, including tag. */ @@ -614,6 +623,18 @@ public final class CodedOutputStream { computeBytesSize(WireFormat.MESSAGE_SET_MESSAGE, value); } + /** + * Compute the number of bytes that would be needed to encode an + * lazily parsed MessageSet extension field to the stream. For + * historical reasons, the wire format differs from normal fields. + */ + public static int computeLazyFieldMessageSetExtensionSize( + final int fieldNumber, final LazyField value) { + return computeTagSize(WireFormat.MESSAGE_SET_ITEM) * 2 + + computeUInt32Size(WireFormat.MESSAGE_SET_TYPE_ID, fieldNumber) + + computeLazyFieldSize(WireFormat.MESSAGE_SET_MESSAGE, value); + } + // ----------------------------------------------------------------- /** @@ -730,6 +751,15 @@ public final class CodedOutputStream { } /** + * Compute the number of bytes that would be needed to encode an embedded + * message stored in lazy field. + */ + public static int computeLazyFieldSizeNoTag(final LazyField value) { + final int size = value.getSerializedSize(); + return computeRawVarint32Size(size) + size; + } + + /** * Compute the number of bytes that would be needed to encode a * {@code bytes} field. */ diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/Descriptors.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/Descriptors.java index 2ee8459..a491305 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/Descriptors.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/Descriptors.java @@ -35,8 +35,10 @@ import com.google.protobuf.DescriptorProtos.*; import java.util.Arrays; import java.util.Collections; import java.util.HashMap; +import java.util.HashSet; import java.util.List; import java.util.Map; +import java.util.Set; import java.io.UnsupportedEncodingException; /** @@ -106,6 +108,11 @@ public final class Descriptors { return Collections.unmodifiableList(Arrays.asList(dependencies)); } + /** Get a list of this file's public dependencies (public imports). */ + public List<FileDescriptor> getPublicDependencies() { + return Collections.unmodifiableList(Arrays.asList(publicDependencies)); + } + /** * Find a message type in the file by name. Does not find nested types. * @@ -216,7 +223,7 @@ public final class Descriptors { public static FileDescriptor buildFrom(final FileDescriptorProto proto, final FileDescriptor[] dependencies) throws DescriptorValidationException { - // Building decsriptors involves two steps: translating and linking. + // Building descriptors involves two steps: translating and linking. // In the translation step (implemented by FileDescriptor's // constructor), we build an object tree mirroring the // FileDescriptorProto's tree and put all of the descriptors into the @@ -317,12 +324,12 @@ public final class Descriptors { * {@link FileDescriptor#internalBuildGeneratedFileFrom}, the caller * provides a callback implementing this interface. The callback is called * after the FileDescriptor has been constructed, in order to assign all - * the global variales defined in the generated code which point at parts + * the global variables defined in the generated code which point at parts * of the FileDescriptor. The callback returns an ExtensionRegistry which * contains any extensions which might be used in the descriptor -- that * is, extensions of the various "Options" messages defined in * descriptor.proto. The callback may also return null to indicate that - * no extensions are used in the decsriptor. + * no extensions are used in the descriptor. */ public interface InternalDescriptorAssigner { ExtensionRegistry assignDescriptors(FileDescriptor root); @@ -334,6 +341,7 @@ public final class Descriptors { private final ServiceDescriptor[] services; private final FieldDescriptor[] extensions; private final FileDescriptor[] dependencies; + private final FileDescriptor[] publicDependencies; private final DescriptorPool pool; private FileDescriptor(final FileDescriptorProto proto, @@ -343,6 +351,17 @@ public final class Descriptors { this.pool = pool; this.proto = proto; this.dependencies = dependencies.clone(); + this.publicDependencies = + new FileDescriptor[proto.getPublicDependencyCount()]; + for (int i = 0; i < proto.getPublicDependencyCount(); i++) { + int index = proto.getPublicDependency(i); + if (index < 0 || index >= this.dependencies.length) { + throw new DescriptorValidationException(this, + "Invalid public dependency index."); + } + this.publicDependencies[i] = + this.dependencies[proto.getPublicDependency(i)]; + } pool.addPackage(getPackage(), this); @@ -390,7 +409,7 @@ public final class Descriptors { * in the original. This method is needed for bootstrapping when a file * defines custom options. The options may be defined in the file itself, * so we can't actually parse them until we've constructed the descriptors, - * but to construct the decsriptors we have to have parsed the descriptor + * but to construct the descriptors we have to have parsed the descriptor * protos. So, we have to parse the descriptor protos a second time after * constructing the descriptors. */ @@ -641,7 +660,7 @@ public final class Descriptors { FieldSet.FieldDescriptorLite<FieldDescriptor> { /** * Get the index of this descriptor within its parent. - * @see Descriptor#getIndex() + * @see Descriptors.Descriptor#getIndex() */ public int getIndex() { return index; } @@ -656,7 +675,7 @@ public final class Descriptors { /** * Get the field's fully-qualified name. - * @see Descriptor#getFullName() + * @see Descriptors.Descriptor#getFullName() */ public String getFullName() { return fullName; } @@ -943,7 +962,8 @@ public final class Descriptors { private void crossLink() throws DescriptorValidationException { if (proto.hasExtendee()) { final GenericDescriptor extendee = - file.pool.lookupSymbol(proto.getExtendee(), this); + file.pool.lookupSymbol(proto.getExtendee(), this, + DescriptorPool.SearchFilter.TYPES_ONLY); if (!(extendee instanceof Descriptor)) { throw new DescriptorValidationException(this, '\"' + proto.getExtendee() + "\" is not a message type."); @@ -960,7 +980,8 @@ public final class Descriptors { if (proto.hasTypeName()) { final GenericDescriptor typeDescriptor = - file.pool.lookupSymbol(proto.getTypeName(), this); + file.pool.lookupSymbol(proto.getTypeName(), this, + DescriptorPool.SearchFilter.TYPES_ONLY); if (!proto.hasType()) { // Choose field type based on symbol. @@ -1149,7 +1170,7 @@ public final class Descriptors { implements GenericDescriptor, Internal.EnumLiteMap<EnumValueDescriptor> { /** * Get the index of this descriptor within its parent. - * @see Descriptor#getIndex() + * @see Descriptors.Descriptor#getIndex() */ public int getIndex() { return index; } @@ -1161,7 +1182,7 @@ public final class Descriptors { /** * Get the type's fully-qualified name. - * @see Descriptor#getFullName() + * @see Descriptors.Descriptor#getFullName() */ public String getFullName() { return fullName; } @@ -1182,7 +1203,7 @@ public final class Descriptors { /** * Find an enum value by name. * @param name The unqualified name of the value (e.g. "FOO"). - * @return the value's decsriptor, or {@code null} if not found. + * @return the value's descriptor, or {@code null} if not found. */ public EnumValueDescriptor findValueByName(final String name) { final GenericDescriptor result = @@ -1198,7 +1219,7 @@ public final class Descriptors { * Find an enum value by number. If multiple enum values have the same * number, this returns the first defined value with that number. * @param number The value's number. - * @return the value's decsriptor, or {@code null} if not found. + * @return the value's descriptor, or {@code null} if not found. */ public EnumValueDescriptor findValueByNumber(final int number) { return file.pool.enumValuesByNumber.get( @@ -1261,7 +1282,7 @@ public final class Descriptors { implements GenericDescriptor, Internal.EnumLite { /** * Get the index of this descriptor within its parent. - * @see Descriptor#getIndex() + * @see Descriptors.Descriptor#getIndex() */ public int getIndex() { return index; } @@ -1276,7 +1297,7 @@ public final class Descriptors { /** * Get the value's fully-qualified name. - * @see Descriptor#getFullName() + * @see Descriptors.Descriptor#getFullName() */ public String getFullName() { return fullName; } @@ -1337,7 +1358,7 @@ public final class Descriptors { /** * Get the type's fully-qualified name. - * @see Descriptor#getFullName() + * @see Descriptors.Descriptor#getFullName() */ public String getFullName() { return fullName; } @@ -1355,7 +1376,7 @@ public final class Descriptors { /** * Find a method by name. * @param name The unqualified name of the method (e.g. "Foo"). - * @return the method's decsriptor, or {@code null} if not found. + * @return the method's descriptor, or {@code null} if not found. */ public MethodDescriptor findMethodByName(final String name) { final GenericDescriptor result = @@ -1427,7 +1448,7 @@ public final class Descriptors { /** * Get the method's fully-qualified name. - * @see Descriptor#getFullName() + * @see Descriptors.Descriptor#getFullName() */ public String getFullName() { return fullName; } @@ -1475,7 +1496,8 @@ public final class Descriptors { private void crossLink() throws DescriptorValidationException { final GenericDescriptor input = - file.pool.lookupSymbol(proto.getInputType(), this); + file.pool.lookupSymbol(proto.getInputType(), this, + DescriptorPool.SearchFilter.TYPES_ONLY); if (!(input instanceof Descriptor)) { throw new DescriptorValidationException(this, '\"' + proto.getInputType() + "\" is not a message type."); @@ -1483,7 +1505,8 @@ public final class Descriptors { inputType = (Descriptor)input; final GenericDescriptor output = - file.pool.lookupSymbol(proto.getOutputType(), this); + file.pool.lookupSymbol(proto.getOutputType(), this, + DescriptorPool.SearchFilter.TYPES_ONLY); if (!(output instanceof Descriptor)) { throw new DescriptorValidationException(this, '\"' + proto.getOutputType() + "\" is not a message type."); @@ -1535,7 +1558,7 @@ public final class Descriptors { public String getProblemSymbolName() { return name; } /** - * Gets the the protocol message representation of the invalid descriptor. + * Gets the protocol message representation of the invalid descriptor. */ public Message getProblemProto() { return proto; } @@ -1590,14 +1613,22 @@ public final class Descriptors { * descriptors defined in a particular file. */ private static final class DescriptorPool { + + /** Defines what subclass of descriptors to search in the descriptor pool. + */ + enum SearchFilter { + TYPES_ONLY, AGGREGATES_ONLY, ALL_SYMBOLS + } + DescriptorPool(final FileDescriptor[] dependencies) { - this.dependencies = new DescriptorPool[dependencies.length]; + this.dependencies = new HashSet<FileDescriptor>(); - for (int i = 0; i < dependencies.length; i++) { - this.dependencies[i] = dependencies[i].pool; + for (int i = 0; i < dependencies.length; i++) { + this.dependencies.add(dependencies[i]); + importPublicDependencies(dependencies[i]); } - for (final FileDescriptor dependency : dependencies) { + for (final FileDescriptor dependency : this.dependencies) { try { addPackage(dependency.getPackage(), dependency); } catch (DescriptorValidationException e) { @@ -1609,7 +1640,16 @@ public final class Descriptors { } } - private final DescriptorPool[] dependencies; + /** Find and put public dependencies of the file into dependencies set.*/ + private void importPublicDependencies(final FileDescriptor file) { + for (FileDescriptor dependency : file.getPublicDependencies()) { + if (dependencies.add(dependency)) { + importPublicDependencies(dependency); + } + } + } + + private final Set<FileDescriptor> dependencies; private final Map<String, GenericDescriptor> descriptorsByName = new HashMap<String, GenericDescriptor>(); @@ -1620,39 +1660,81 @@ public final class Descriptors { /** Find a generic descriptor by fully-qualified name. */ GenericDescriptor findSymbol(final String fullName) { + return findSymbol(fullName, SearchFilter.ALL_SYMBOLS); + } + + /** Find a descriptor by fully-qualified name and given option to only + * search valid field type descriptors. + */ + GenericDescriptor findSymbol(final String fullName, + final SearchFilter filter) { GenericDescriptor result = descriptorsByName.get(fullName); if (result != null) { - return result; + if ((filter==SearchFilter.ALL_SYMBOLS) || + ((filter==SearchFilter.TYPES_ONLY) && isType(result)) || + ((filter==SearchFilter.AGGREGATES_ONLY) && isAggregate(result))) { + return result; + } } - for (final DescriptorPool dependency : dependencies) { - result = dependency.descriptorsByName.get(fullName); + for (final FileDescriptor dependency : dependencies) { + result = dependency.pool.descriptorsByName.get(fullName); if (result != null) { - return result; + if ((filter==SearchFilter.ALL_SYMBOLS) || + ((filter==SearchFilter.TYPES_ONLY) && isType(result)) || + ((filter==SearchFilter.AGGREGATES_ONLY) && isAggregate(result))) { + return result; + } } } return null; } + /** Checks if the descriptor is a valid type for a message field. */ + boolean isType(GenericDescriptor descriptor) { + return (descriptor instanceof Descriptor) || + (descriptor instanceof EnumDescriptor); + } + + /** Checks if the descriptor is a valid namespace type. */ + boolean isAggregate(GenericDescriptor descriptor) { + return (descriptor instanceof Descriptor) || + (descriptor instanceof EnumDescriptor) || + (descriptor instanceof PackageDescriptor) || + (descriptor instanceof ServiceDescriptor); + } + /** - * Look up a descriptor by name, relative to some other descriptor. + * Look up a type descriptor by name, relative to some other descriptor. * The name may be fully-qualified (with a leading '.'), * partially-qualified, or unqualified. C++-like name lookup semantics * are used to search for the matching descriptor. */ GenericDescriptor lookupSymbol(final String name, - final GenericDescriptor relativeTo) + final GenericDescriptor relativeTo, + final DescriptorPool.SearchFilter filter) throws DescriptorValidationException { // TODO(kenton): This could be optimized in a number of ways. GenericDescriptor result; if (name.startsWith(".")) { // Fully-qualified name. - result = findSymbol(name.substring(1)); + result = findSymbol(name.substring(1), filter); } else { // If "name" is a compound identifier, we want to search for the // first component of it, then search within it for the rest. + // If name is something like "Foo.Bar.baz", and symbols named "Foo" are + // defined in multiple parent scopes, we only want to find "Bar.baz" in + // the innermost one. E.g., the following should produce an error: + // message Bar { message Baz {} } + // message Foo { + // message Bar { + // } + // optional Bar.Baz baz = 1; + // } + // So, we look for just "Foo" first, then look for "Bar.baz" within it + // if found. final int firstPartLength = name.indexOf('.'); final String firstPart; if (firstPartLength == -1) { @@ -1670,14 +1752,15 @@ public final class Descriptors { // Chop off the last component of the scope. final int dotpos = scopeToTry.lastIndexOf("."); if (dotpos == -1) { - result = findSymbol(name); + result = findSymbol(name, filter); break; } else { scopeToTry.setLength(dotpos + 1); - // Append firstPart and try to find. + // Append firstPart and try to find scopeToTry.append(firstPart); - result = findSymbol(scopeToTry.toString()); + result = findSymbol(scopeToTry.toString(), + DescriptorPool.SearchFilter.AGGREGATES_ONLY); if (result != null) { if (firstPartLength != -1) { @@ -1686,7 +1769,7 @@ public final class Descriptors { // searching parent scopes. scopeToTry.setLength(dotpos + 1); scopeToTry.append(name); - result = findSymbol(scopeToTry.toString()); + result = findSymbol(scopeToTry.toString(), filter); } break; } @@ -1817,7 +1900,7 @@ public final class Descriptors { /** * Adds a field to the fieldsByNumber table. Throws an exception if a - * field with hte same containing type and number already exists. + * field with the same containing type and number already exists. */ void addFieldByNumber(final FieldDescriptor field) throws DescriptorValidationException { diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/DynamicMessage.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/DynamicMessage.java index c106b66..c0c9fc9 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/DynamicMessage.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/DynamicMessage.java @@ -35,6 +35,7 @@ import com.google.protobuf.Descriptors.FieldDescriptor; import java.io.InputStream; import java.io.IOException; +import java.util.Collections; import java.util.Map; /** @@ -160,7 +161,9 @@ public final class DynamicMessage extends AbstractMessage { verifyContainingType(field); Object result = fields.getField(field); if (result == null) { - if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { + if (field.isRepeated()) { + result = Collections.emptyList(); + } else if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { result = getDefaultInstance(field.getMessageType()); } else { result = field.getDefaultValue(); @@ -198,10 +201,12 @@ public final class DynamicMessage extends AbstractMessage { return fields.isInitialized(); } + @Override public boolean isInitialized() { return isInitialized(type, fields); } + @Override public void writeTo(CodedOutputStream output) throws IOException { if (type.getOptions().getMessageSetWireFormat()) { fields.writeMessageSetTo(output); @@ -212,6 +217,7 @@ public final class DynamicMessage extends AbstractMessage { } } + @Override public int getSerializedSize() { int size = memoizedSize; if (size != -1) return size; @@ -236,6 +242,26 @@ public final class DynamicMessage extends AbstractMessage { return newBuilderForType().mergeFrom(this); } + public Parser<DynamicMessage> getParserForType() { + return new AbstractParser<DynamicMessage>() { + public DynamicMessage parsePartialFrom( + CodedInputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + Builder builder = newBuilder(type); + try { + builder.mergeFrom(input, extensionRegistry); + } catch (InvalidProtocolBufferException e) { + throw e.setUnfinishedMessage(builder.buildPartial()); + } catch (IOException e) { + throw new InvalidProtocolBufferException(e.getMessage()) + .setUnfinishedMessage(builder.buildPartial()); + } + return builder.buildPartial(); + } + }; + } + /** Verifies that the field is a field of this message. */ private void verifyContainingType(FieldDescriptor field) { if (field.getContainingType() != type) { @@ -264,14 +290,18 @@ public final class DynamicMessage extends AbstractMessage { // --------------------------------------------------------------- // Implementation of Message.Builder interface. + @Override public Builder clear() { - if (fields == null) { - throw new IllegalStateException("Cannot call clear() after build()."); + if (fields.isImmutable()) { + fields = FieldSet.newFieldSet(); + } else { + fields.clear(); } - fields.clear(); + unknownFields = UnknownFieldSet.getDefaultInstance(); return this; } + @Override public Builder mergeFrom(Message other) { if (other instanceof DynamicMessage) { // This should be somewhat faster than calling super.mergeFrom(). @@ -280,6 +310,7 @@ public final class DynamicMessage extends AbstractMessage { throw new IllegalArgumentException( "mergeFrom(Message) can only merge messages of the same type."); } + ensureIsMutable(); fields.mergeFrom(otherDynamicMessage.fields); mergeUnknownFields(otherDynamicMessage.unknownFields); return this; @@ -289,8 +320,7 @@ public final class DynamicMessage extends AbstractMessage { } public DynamicMessage build() { - // If fields == null, we'll throw an appropriate exception later. - if (fields != null && !isInitialized()) { + if (!isInitialized()) { throw newUninitializedMessageException( new DynamicMessage(type, fields, unknownFields)); } @@ -312,21 +342,17 @@ public final class DynamicMessage extends AbstractMessage { } public DynamicMessage buildPartial() { - if (fields == null) { - throw new IllegalStateException( - "build() has already been called on this Builder."); - } fields.makeImmutable(); DynamicMessage result = new DynamicMessage(type, fields, unknownFields); - fields = null; - unknownFields = null; return result; } + @Override public Builder clone() { Builder result = new Builder(type); result.fields.mergeFrom(fields); + result.mergeUnknownFields(unknownFields); return result; } @@ -377,12 +403,14 @@ public final class DynamicMessage extends AbstractMessage { public Builder setField(FieldDescriptor field, Object value) { verifyContainingType(field); + ensureIsMutable(); fields.setField(field, value); return this; } public Builder clearField(FieldDescriptor field) { verifyContainingType(field); + ensureIsMutable(); fields.clearField(field); return this; } @@ -400,12 +428,14 @@ public final class DynamicMessage extends AbstractMessage { public Builder setRepeatedField(FieldDescriptor field, int index, Object value) { verifyContainingType(field); + ensureIsMutable(); fields.setRepeatedField(field, index, value); return this; } public Builder addRepeatedField(FieldDescriptor field, Object value) { verifyContainingType(field); + ensureIsMutable(); fields.addRepeatedField(field, value); return this; } @@ -419,6 +449,7 @@ public final class DynamicMessage extends AbstractMessage { return this; } + @Override public Builder mergeUnknownFields(UnknownFieldSet unknownFields) { this.unknownFields = UnknownFieldSet.newBuilder(this.unknownFields) @@ -434,5 +465,18 @@ public final class DynamicMessage extends AbstractMessage { "FieldDescriptor does not match message type."); } } + + private void ensureIsMutable() { + if (fields.isImmutable()) { + fields = fields.clone(); + } + } + + @Override + public com.google.protobuf.Message.Builder getFieldBuilder(FieldDescriptor field) { + // TODO(xiangl): need implementation for dynamic message + throw new UnsupportedOperationException( + "getFieldBuilder() called on a dynamic message type."); + } } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/ExtensionRegistryLite.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/ExtensionRegistryLite.java index d5288dd..1e1289d 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/ExtensionRegistryLite.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/ExtensionRegistryLite.java @@ -43,7 +43,7 @@ import java.util.Map; * make sense to mix the two, since if you have any regular types in your * program, you then require the full runtime and lose all the benefits of * the lite runtime, so you might as well make all your types be regular types. - * However, in some cases (e.g. when depending on multiple third-patry libraries + * However, in some cases (e.g. when depending on multiple third-party libraries * where one uses lite types and one uses regular), you may find yourself * wanting to mix the two. In this case things get more complicated. * <p> @@ -71,6 +71,22 @@ import java.util.Map; * @author kenton@google.com Kenton Varda */ public class ExtensionRegistryLite { + + // Set true to enable lazy parsing feature for MessageSet. + // + // TODO(xiangl): Now we use a global flag to control whether enable lazy + // parsing feature for MessageSet, which may be too crude for some + // applications. Need to support this feature on smaller granularity. + private static volatile boolean eagerlyParseMessageSets = false; + + public static boolean isEagerlyParseMessageSets() { + return eagerlyParseMessageSets; + } + + public static void setEagerlyParseMessageSets(boolean isEagerlyParse) { + eagerlyParseMessageSets = isEagerlyParse; + } + /** Construct a new, empty instance. */ public static ExtensionRegistryLite newInstance() { return new ExtensionRegistryLite(); diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/FieldSet.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/FieldSet.java index a85dbaa..2663694 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/FieldSet.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/FieldSet.java @@ -30,12 +30,14 @@ package com.google.protobuf; +import com.google.protobuf.LazyField.LazyIterator; + +import java.io.IOException; import java.util.ArrayList; import java.util.Collections; import java.util.Iterator; import java.util.List; import java.util.Map; -import java.io.IOException; /** * A class which represents an arbitrary set of fields of some message type. @@ -68,6 +70,7 @@ final class FieldSet<FieldDescriptorType extends private final SmallSortedMap<FieldDescriptorType, Object> fields; private boolean isImmutable; + private boolean hasLazyField = false; /** Construct a new FieldSet. */ private FieldSet() { @@ -95,7 +98,7 @@ final class FieldSet<FieldDescriptorType extends FieldSet<T> emptySet() { return DEFAULT_INSTANCE; } - @SuppressWarnings("unchecked") + @SuppressWarnings("rawtypes") private static final FieldSet DEFAULT_INSTANCE = new FieldSet(true); /** Make this FieldSet immutable from this point forward. */ @@ -109,7 +112,7 @@ final class FieldSet<FieldDescriptorType extends } /** - * Retuns whether the FieldSet is immutable. This is true if it is the + * Returns whether the FieldSet is immutable. This is true if it is the * {@link #emptySet} or if {@link #makeImmutable} were called. * * @return whether the FieldSet is immutable. @@ -139,6 +142,7 @@ final class FieldSet<FieldDescriptorType extends FieldDescriptorType descriptor = entry.getKey(); clone.setField(descriptor, entry.getValue()); } + clone.hasLazyField = hasLazyField; return clone; } @@ -147,21 +151,52 @@ final class FieldSet<FieldDescriptorType extends /** See {@link Message.Builder#clear()}. */ public void clear() { fields.clear(); + hasLazyField = false; } /** * Get a simple map containing all the fields. */ public Map<FieldDescriptorType, Object> getAllFields() { + if (hasLazyField) { + SmallSortedMap<FieldDescriptorType, Object> result = + SmallSortedMap.newFieldMap(16); + for (int i = 0; i < fields.getNumArrayEntries(); i++) { + cloneFieldEntry(result, fields.getArrayEntryAt(i)); + } + for (Map.Entry<FieldDescriptorType, Object> entry : + fields.getOverflowEntries()) { + cloneFieldEntry(result, entry); + } + if (fields.isImmutable()) { + result.makeImmutable(); + } + return result; + } return fields.isImmutable() ? fields : Collections.unmodifiableMap(fields); } + private void cloneFieldEntry(Map<FieldDescriptorType, Object> map, + Map.Entry<FieldDescriptorType, Object> entry) { + FieldDescriptorType key = entry.getKey(); + Object value = entry.getValue(); + if (value instanceof LazyField) { + map.put(key, ((LazyField) value).getValue()); + } else { + map.put(key, value); + } + } + /** * Get an iterator to the field map. This iterator should not be leaked out - * of the protobuf library as it is not protected from mutation when - * fields is not immutable. + * of the protobuf library as it is not protected from mutation when fields + * is not immutable. */ public Iterator<Map.Entry<FieldDescriptorType, Object>> iterator() { + if (hasLazyField) { + return new LazyIterator<FieldDescriptorType>( + fields.entrySet().iterator()); + } return fields.entrySet().iterator(); } @@ -185,14 +220,18 @@ final class FieldSet<FieldDescriptorType extends * to the caller to fetch the field's default value. */ public Object getField(final FieldDescriptorType descriptor) { - return fields.get(descriptor); + Object o = fields.get(descriptor); + if (o instanceof LazyField) { + return ((LazyField) o).getValue(); + } + return o; } /** * Useful for implementing * {@link Message.Builder#setField(Descriptors.FieldDescriptor,Object)}. */ - @SuppressWarnings("unchecked") + @SuppressWarnings({"unchecked", "rawtypes"}) public void setField(final FieldDescriptorType descriptor, Object value) { if (descriptor.isRepeated()) { @@ -204,7 +243,7 @@ final class FieldSet<FieldDescriptorType extends // Wrap the contents in a new list so that the caller cannot change // the list's contents after setting it. final List newList = new ArrayList(); - newList.addAll((List)value); + newList.addAll((List) value); for (final Object element : newList) { verifyType(descriptor.getLiteType(), element); } @@ -213,6 +252,9 @@ final class FieldSet<FieldDescriptorType extends verifyType(descriptor.getLiteType(), value); } + if (value instanceof LazyField) { + hasLazyField = true; + } fields.put(descriptor, value); } @@ -222,6 +264,9 @@ final class FieldSet<FieldDescriptorType extends */ public void clearField(final FieldDescriptorType descriptor) { fields.remove(descriptor); + if (fields.isEmpty()) { + hasLazyField = false; + } } /** @@ -234,7 +279,7 @@ final class FieldSet<FieldDescriptorType extends "getRepeatedField() can only be called on repeated fields."); } - final Object value = fields.get(descriptor); + final Object value = getField(descriptor); if (value == null) { return 0; } else { @@ -253,7 +298,7 @@ final class FieldSet<FieldDescriptorType extends "getRepeatedField() can only be called on repeated fields."); } - final Object value = fields.get(descriptor); + final Object value = getField(descriptor); if (value == null) { throw new IndexOutOfBoundsException(); @@ -275,13 +320,13 @@ final class FieldSet<FieldDescriptorType extends "getRepeatedField() can only be called on repeated fields."); } - final Object list = fields.get(descriptor); + final Object list = getField(descriptor); if (list == null) { throw new IndexOutOfBoundsException(); } verifyType(descriptor.getLiteType(), value); - ((List) list).set(index, value); + ((List<Object>) list).set(index, value); } /** @@ -298,13 +343,13 @@ final class FieldSet<FieldDescriptorType extends verifyType(descriptor.getLiteType(), value); - final Object existingValue = fields.get(descriptor); - List list; + final Object existingValue = getField(descriptor); + List<Object> list; if (existingValue == null) { - list = new ArrayList(); + list = new ArrayList<Object>(); fields.put(descriptor, list); } else { - list = (List) existingValue; + list = (List<Object>) existingValue; } list.add(value); @@ -338,7 +383,8 @@ final class FieldSet<FieldDescriptorType extends break; case MESSAGE: // TODO(kenton): Caller must do type checking here, I guess. - isValid = value instanceof MessageLite; + isValid = + (value instanceof MessageLite) || (value instanceof LazyField); break; } @@ -392,8 +438,16 @@ final class FieldSet<FieldDescriptorType extends } } } else { - if (!((MessageLite) entry.getValue()).isInitialized()) { - return false; + Object value = entry.getValue(); + if (value instanceof MessageLite) { + if (!((MessageLite) value).isInitialized()) { + return false; + } + } else if (value instanceof LazyField) { + return true; + } else { + throw new IllegalArgumentException( + "Wrong object type used with protocol message reflection."); } } } @@ -416,7 +470,8 @@ final class FieldSet<FieldDescriptorType extends } /** - * Like {@link #mergeFrom(Message)}, but merges from another {@link FieldSet}. + * Like {@link Message.Builder#mergeFrom(Message)}, but merges from another + * {@link FieldSet}. */ public void mergeFrom(final FieldSet<FieldDescriptorType> other) { for (int i = 0; i < other.fields.getNumArrayEntries(); i++) { @@ -428,14 +483,17 @@ final class FieldSet<FieldDescriptorType extends } } - @SuppressWarnings("unchecked") + @SuppressWarnings({"unchecked", "rawtypes"}) private void mergeFromField( final Map.Entry<FieldDescriptorType, Object> entry) { final FieldDescriptorType descriptor = entry.getKey(); - final Object otherValue = entry.getValue(); + Object otherValue = entry.getValue(); + if (otherValue instanceof LazyField) { + otherValue = ((LazyField) otherValue).getValue(); + } if (descriptor.isRepeated()) { - Object value = fields.get(descriptor); + Object value = getField(descriptor); if (value == null) { // Our list is empty, but we still need to make a defensive copy of // the other list since we don't know if the other FieldSet is still @@ -446,7 +504,7 @@ final class FieldSet<FieldDescriptorType extends ((List) value).addAll((List) otherValue); } } else if (descriptor.getLiteJavaType() == WireFormat.JavaType.MESSAGE) { - Object value = fields.get(descriptor); + Object value = getField(descriptor); if (value == null) { fields.put(descriptor, otherValue); } else { @@ -457,7 +515,6 @@ final class FieldSet<FieldDescriptorType extends ((MessageLite) value).toBuilder(), (MessageLite) otherValue) .build()); } - } else { fields.put(descriptor, otherValue); } @@ -646,7 +703,11 @@ final class FieldSet<FieldDescriptorType extends } } } else { - writeElement(output, type, number, value); + if (value instanceof LazyField) { + writeElement(output, type, number, ((LazyField) value).getValue()); + } else { + writeElement(output, type, number, value); + } } } @@ -686,12 +747,18 @@ final class FieldSet<FieldDescriptorType extends private int getMessageSetSerializedSize( final Map.Entry<FieldDescriptorType, Object> entry) { final FieldDescriptorType descriptor = entry.getKey(); - if (descriptor.getLiteJavaType() == WireFormat.JavaType.MESSAGE && - !descriptor.isRepeated() && !descriptor.isPacked()) { - return CodedOutputStream.computeMessageSetExtensionSize( - entry.getKey().getNumber(), (MessageLite) entry.getValue()); + Object value = entry.getValue(); + if (descriptor.getLiteJavaType() == WireFormat.JavaType.MESSAGE + && !descriptor.isRepeated() && !descriptor.isPacked()) { + if (value instanceof LazyField) { + return CodedOutputStream.computeLazyFieldMessageSetExtensionSize( + entry.getKey().getNumber(), (LazyField) value); + } else { + return CodedOutputStream.computeMessageSetExtensionSize( + entry.getKey().getNumber(), (MessageLite) value); + } } else { - return computeFieldSize(descriptor, entry.getValue()); + return computeFieldSize(descriptor, value); } } @@ -741,7 +808,6 @@ final class FieldSet<FieldDescriptorType extends case BOOL : return CodedOutputStream.computeBoolSizeNoTag ((Boolean )value); case STRING : return CodedOutputStream.computeStringSizeNoTag ((String )value); case GROUP : return CodedOutputStream.computeGroupSizeNoTag ((MessageLite)value); - case MESSAGE : return CodedOutputStream.computeMessageSizeNoTag ((MessageLite)value); case BYTES : return CodedOutputStream.computeBytesSizeNoTag ((ByteString )value); case UINT32 : return CodedOutputStream.computeUInt32SizeNoTag ((Integer )value); case SFIXED32: return CodedOutputStream.computeSFixed32SizeNoTag((Integer )value); @@ -749,6 +815,13 @@ final class FieldSet<FieldDescriptorType extends case SINT32 : return CodedOutputStream.computeSInt32SizeNoTag ((Integer )value); case SINT64 : return CodedOutputStream.computeSInt64SizeNoTag ((Long )value); + case MESSAGE: + if (value instanceof LazyField) { + return CodedOutputStream.computeLazyFieldSizeNoTag((LazyField) value); + } else { + return CodedOutputStream.computeMessageSizeNoTag((MessageLite) value); + } + case ENUM: return CodedOutputStream.computeEnumSizeNoTag( ((Internal.EnumLite) value).getNumber()); diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessage.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessage.java index b5eaded..0c15ca8 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessage.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessage.java @@ -58,8 +58,6 @@ public abstract class GeneratedMessage extends AbstractMessage implements Serializable { private static final long serialVersionUID = 1L; - private final UnknownFieldSet unknownFields; - /** * For testing. Allows a test to disable the optimization that avoids using * field builders for nested messages until they are requested. By disabling @@ -68,11 +66,14 @@ public abstract class GeneratedMessage extends AbstractMessage protected static boolean alwaysUseFieldBuilders = false; protected GeneratedMessage() { - this.unknownFields = UnknownFieldSet.getDefaultInstance(); } protected GeneratedMessage(Builder<?> builder) { - this.unknownFields = builder.getUnknownFields(); + } + + public Parser<? extends Message> getParserForType() { + throw new UnsupportedOperationException( + "This is supposed to be overridden by subclasses."); } /** @@ -175,8 +176,28 @@ public abstract class GeneratedMessage extends AbstractMessage } //@Override (Java 1.6 override semantics, but we must support 1.5) - public final UnknownFieldSet getUnknownFields() { - return unknownFields; + public UnknownFieldSet getUnknownFields() { + throw new UnsupportedOperationException( + "This is supposed to be overridden by subclasses."); + } + + /** + * Called by subclasses to parse an unknown field. + * @return {@code true} unless the tag is an end-group tag. + */ + protected boolean parseUnknownField( + CodedInputStream input, + UnknownFieldSet.Builder unknownFields, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + return unknownFields.mergeFieldFrom(tag, input); + } + + /** + * Used by parsing constructors in generated classes. + */ + protected void makeExtensionsImmutable() { + // Noop for messages without extensions. } protected abstract Message.Builder newBuilderForType(BuilderParent parent); @@ -319,6 +340,11 @@ public abstract class GeneratedMessage extends AbstractMessage } //@Override (Java 1.6 override semantics, but we must support 1.5) + public Message.Builder getFieldBuilder(final FieldDescriptor field) { + return internalGetFieldAccessorTable().getField(field).getBuilder(this); + } + + //@Override (Java 1.6 override semantics, but we must support 1.5) public boolean hasField(final FieldDescriptor field) { return internalGetFieldAccessorTable().getField(field).has(this); } @@ -626,6 +652,25 @@ public abstract class GeneratedMessage extends AbstractMessage return super.isInitialized() && extensionsAreInitialized(); } + @Override + protected boolean parseUnknownField( + CodedInputStream input, + UnknownFieldSet.Builder unknownFields, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + return AbstractMessage.Builder.mergeFieldFrom( + input, unknownFields, extensionRegistry, getDescriptorForType(), + null, extensions, tag); + } + + /** + * Used by parsing constructors in generated classes. + */ + @Override + protected void makeExtensionsImmutable() { + extensions.makeImmutable(); + } + /** * Used by subclasses to serialize extensions. Extension ranges may be * interleaved with field numbers, but we must write them in canonical @@ -655,9 +700,21 @@ public abstract class GeneratedMessage extends AbstractMessage if (messageSetWireFormat && descriptor.getLiteJavaType() == WireFormat.JavaType.MESSAGE && !descriptor.isRepeated()) { - output.writeMessageSetExtension(descriptor.getNumber(), - (Message) next.getValue()); + if (next instanceof LazyField.LazyEntry<?>) { + output.writeRawMessageSetExtension(descriptor.getNumber(), + ((LazyField.LazyEntry<?>) next).getField().toByteString()); + } else { + output.writeMessageSetExtension(descriptor.getNumber(), + (Message) next.getValue()); + } } else { + // TODO(xiangl): Taken care of following code, it may cause + // problem when we use LazyField for normal fields/extensions. + // Due to the optional field can be duplicated at the end of + // serialized bytes, which will make the serialized size change + // after lazy field parsed. So when we use LazyField globally, + // we need to change the following write method to write cached + // bytes directly rather than write the parsed message. FieldSet.writeField(descriptor, next.getValue(), output); } if (iter.hasNext()) { @@ -974,7 +1031,8 @@ public abstract class GeneratedMessage extends AbstractMessage final ExtensionRegistryLite extensionRegistry, final int tag) throws IOException { return AbstractMessage.Builder.mergeFieldFrom( - input, unknownFields, extensionRegistry, this, tag); + input, unknownFields, extensionRegistry, getDescriptorForType(), + this, null, tag); } // --------------------------------------------------------------- @@ -1405,39 +1463,72 @@ public abstract class GeneratedMessage extends AbstractMessage final String[] camelCaseNames, final Class<? extends GeneratedMessage> messageClass, final Class<? extends Builder> builderClass) { + this(descriptor, camelCaseNames); + ensureFieldAccessorsInitialized(messageClass, builderClass); + } + + /** + * Construct a FieldAccessorTable for a particular message class without + * initializing FieldAccessors. + */ + public FieldAccessorTable( + final Descriptor descriptor, + final String[] camelCaseNames) { this.descriptor = descriptor; + this.camelCaseNames = camelCaseNames; fields = new FieldAccessor[descriptor.getFields().size()]; + initialized = false; + } - for (int i = 0; i < fields.length; i++) { - final FieldDescriptor field = descriptor.getFields().get(i); - if (field.isRepeated()) { - if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { - fields[i] = new RepeatedMessageFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); - } else if (field.getJavaType() == FieldDescriptor.JavaType.ENUM) { - fields[i] = new RepeatedEnumFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); - } else { - fields[i] = new RepeatedFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); - } - } else { - if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { - fields[i] = new SingularMessageFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); - } else if (field.getJavaType() == FieldDescriptor.JavaType.ENUM) { - fields[i] = new SingularEnumFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); + /** + * Ensures the field accessors are initialized. This method is thread-safe. + * + * @param messageClass The message type. + * @param builderClass The builder type. + * @return this + */ + public FieldAccessorTable ensureFieldAccessorsInitialized( + Class<? extends GeneratedMessage> messageClass, + Class<? extends Builder> builderClass) { + if (initialized) { return this; } + synchronized (this) { + if (initialized) { return this; } + for (int i = 0; i < fields.length; i++) { + FieldDescriptor field = descriptor.getFields().get(i); + if (field.isRepeated()) { + if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { + fields[i] = new RepeatedMessageFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } else if (field.getJavaType() == FieldDescriptor.JavaType.ENUM) { + fields[i] = new RepeatedEnumFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } else { + fields[i] = new RepeatedFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } } else { - fields[i] = new SingularFieldAccessor( - field, camelCaseNames[i], messageClass, builderClass); + if (field.getJavaType() == FieldDescriptor.JavaType.MESSAGE) { + fields[i] = new SingularMessageFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } else if (field.getJavaType() == FieldDescriptor.JavaType.ENUM) { + fields[i] = new SingularEnumFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } else { + fields[i] = new SingularFieldAccessor( + field, camelCaseNames[i], messageClass, builderClass); + } } } + initialized = true; + camelCaseNames = null; + return this; } } private final Descriptor descriptor; private final FieldAccessor[] fields; + private String[] camelCaseNames; + private volatile boolean initialized; /** Get the FieldAccessor for a particular field. */ private FieldAccessor getField(final FieldDescriptor field) { @@ -1472,6 +1563,7 @@ public abstract class GeneratedMessage extends AbstractMessage int getRepeatedCount(GeneratedMessage.Builder builder); void clear(Builder builder); Message.Builder newBuilder(); + Message.Builder getBuilder(GeneratedMessage.Builder builder); } // --------------------------------------------------------------- @@ -1551,6 +1643,10 @@ public abstract class GeneratedMessage extends AbstractMessage throw new UnsupportedOperationException( "newBuilderForField() called on a non-Message type."); } + public Message.Builder getBuilder(GeneratedMessage.Builder builder) { + throw new UnsupportedOperationException( + "getFieldBuilder() called on a non-Message type."); + } } private static class RepeatedFieldAccessor implements FieldAccessor { @@ -1573,8 +1669,6 @@ public abstract class GeneratedMessage extends AbstractMessage "get" + camelCaseName + "List"); getMethodBuilder = getMethodOrDie(builderClass, "get" + camelCaseName + "List"); - - getRepeatedMethod = getMethodOrDie(messageClass, "get" + camelCaseName, Integer.TYPE); getRepeatedMethodBuilder = @@ -1625,11 +1719,11 @@ public abstract class GeneratedMessage extends AbstractMessage } public boolean has(final GeneratedMessage message) { throw new UnsupportedOperationException( - "hasField() called on a singular field."); + "hasField() called on a repeated field."); } public boolean has(GeneratedMessage.Builder builder) { throw new UnsupportedOperationException( - "hasField() called on a singular field."); + "hasField() called on a repeated field."); } public int getRepeatedCount(final GeneratedMessage message) { return (Integer) invokeOrDie(getCountMethod, message); @@ -1644,6 +1738,10 @@ public abstract class GeneratedMessage extends AbstractMessage throw new UnsupportedOperationException( "newBuilderForField() called on a non-Message type."); } + public Message.Builder getBuilder(GeneratedMessage.Builder builder) { + throw new UnsupportedOperationException( + "getFieldBuilder() called on a non-Message type."); + } } // --------------------------------------------------------------- @@ -1753,9 +1851,12 @@ public abstract class GeneratedMessage extends AbstractMessage super(descriptor, camelCaseName, messageClass, builderClass); newBuilderMethod = getMethodOrDie(type, "newBuilder"); + getBuilderMethodBuilder = + getMethodOrDie(builderClass, "get" + camelCaseName + "Builder"); } private final Method newBuilderMethod; + private final Method getBuilderMethodBuilder; private Object coerceType(final Object value) { if (type.isInstance(value)) { @@ -1766,7 +1867,7 @@ public abstract class GeneratedMessage extends AbstractMessage // DynamicMessage -- we should accept it. In this case we can make // a copy of the message. return ((Message.Builder) invokeOrDie(newBuilderMethod, null)) - .mergeFrom((Message) value).build(); + .mergeFrom((Message) value).buildPartial(); } } @@ -1778,6 +1879,10 @@ public abstract class GeneratedMessage extends AbstractMessage public Message.Builder newBuilder() { return (Message.Builder) invokeOrDie(newBuilderMethod, null); } + @Override + public Message.Builder getBuilder(GeneratedMessage.Builder builder) { + return (Message.Builder) invokeOrDie(getBuilderMethodBuilder, builder); + } } private static final class RepeatedMessageFieldAccessor @@ -1825,7 +1930,7 @@ public abstract class GeneratedMessage extends AbstractMessage /** * Replaces this object in the output stream with a serialized form. * Part of Java's serialization magic. Generated sub-classes must override - * this method by calling <code>return super.writeReplace();</code> + * this method by calling {@code return super.writeReplace();} * @return a SerializedForm of this message */ protected Object writeReplace() throws ObjectStreamException { diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessageLite.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessageLite.java index 1813e9b..437e341 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessageLite.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/GeneratedMessageLite.java @@ -55,6 +55,29 @@ public abstract class GeneratedMessageLite extends AbstractMessageLite protected GeneratedMessageLite(Builder builder) { } + public Parser<? extends MessageLite> getParserForType() { + throw new UnsupportedOperationException( + "This is supposed to be overridden by subclasses."); + } + + /** + * Called by subclasses to parse an unknown field. + * @return {@code true} unless the tag is an end-group tag. + */ + protected boolean parseUnknownField( + CodedInputStream input, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + return input.skipField(tag); + } + + /** + * Used by parsing constructors in generated classes. + */ + protected void makeExtensionsImmutable() { + // Noop for messages without extensions. + } + @SuppressWarnings("unchecked") public abstract static class Builder<MessageType extends GeneratedMessageLite, BuilderType extends Builder> @@ -86,9 +109,9 @@ public abstract class GeneratedMessageLite extends AbstractMessageLite * @return {@code true} unless the tag is an end-group tag. */ protected boolean parseUnknownField( - final CodedInputStream input, - final ExtensionRegistryLite extensionRegistry, - final int tag) throws IOException { + CodedInputStream input, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { return input.skipField(tag); } } @@ -194,6 +217,31 @@ public abstract class GeneratedMessageLite extends AbstractMessageLite } /** + * Called by subclasses to parse an unknown field or an extension. + * @return {@code true} unless the tag is an end-group tag. + */ + @Override + protected boolean parseUnknownField( + CodedInputStream input, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + return GeneratedMessageLite.parseUnknownField( + extensions, + getDefaultInstanceForType(), + input, + extensionRegistry, + tag); + } + + /** + * Used by parsing constructors in generated classes. + */ + @Override + protected void makeExtensionsImmutable() { + extensions.makeImmutable(); + } + + /** * Used by subclasses to serialize extensions. Extension ranges may be * interleaved with field numbers, but we must write them in canonical * (sorted by field number) order. ExtensionWriter helps us write @@ -400,121 +448,139 @@ public abstract class GeneratedMessageLite extends AbstractMessageLite */ @Override protected boolean parseUnknownField( - final CodedInputStream input, - final ExtensionRegistryLite extensionRegistry, - final int tag) throws IOException { - final int wireType = WireFormat.getTagWireType(tag); - final int fieldNumber = WireFormat.getTagFieldNumber(tag); - - final GeneratedExtension<MessageType, ?> extension = - extensionRegistry.findLiteExtensionByNumber( - getDefaultInstanceForType(), fieldNumber); - - boolean unknown = false; - boolean packed = false; - if (extension == null) { - unknown = true; // Unknown field. - } else if (wireType == FieldSet.getWireFormatForFieldType( - extension.descriptor.getLiteType(), - false /* isPacked */)) { - packed = false; // Normal, unpacked value. - } else if (extension.descriptor.isRepeated && - extension.descriptor.type.isPackable() && - wireType == FieldSet.getWireFormatForFieldType( - extension.descriptor.getLiteType(), - true /* isPacked */)) { - packed = true; // Packed value. - } else { - unknown = true; // Wrong wire type. - } + CodedInputStream input, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + ensureExtensionsIsMutable(); + return GeneratedMessageLite.parseUnknownField( + extensions, + getDefaultInstanceForType(), + input, + extensionRegistry, + tag); + } - if (unknown) { // Unknown field or wrong wire type. Skip. - return input.skipField(tag); - } + protected final void mergeExtensionFields(final MessageType other) { + ensureExtensionsIsMutable(); + extensions.mergeFrom(((ExtendableMessage) other).extensions); + } + } - if (packed) { - final int length = input.readRawVarint32(); - final int limit = input.pushLimit(length); - if (extension.descriptor.getLiteType() == WireFormat.FieldType.ENUM) { - while (input.getBytesUntilLimit() > 0) { - final int rawValue = input.readEnum(); - final Object value = - extension.descriptor.getEnumType().findValueByNumber(rawValue); - if (value == null) { - // If the number isn't recognized as a valid value for this - // enum, drop it (don't even add it to unknownFields). - return true; - } - ensureExtensionsIsMutable(); - extensions.addRepeatedField(extension.descriptor, value); - } - } else { - while (input.getBytesUntilLimit() > 0) { - final Object value = - FieldSet.readPrimitiveField(input, - extension.descriptor.getLiteType()); - ensureExtensionsIsMutable(); - extensions.addRepeatedField(extension.descriptor, value); + // ----------------------------------------------------------------- + + /** + * Parse an unknown field or an extension. + * @return {@code true} unless the tag is an end-group tag. + */ + private static <MessageType extends MessageLite> + boolean parseUnknownField( + FieldSet<ExtensionDescriptor> extensions, + MessageType defaultInstance, + CodedInputStream input, + ExtensionRegistryLite extensionRegistry, + int tag) throws IOException { + int wireType = WireFormat.getTagWireType(tag); + int fieldNumber = WireFormat.getTagFieldNumber(tag); + + GeneratedExtension<MessageType, ?> extension = + extensionRegistry.findLiteExtensionByNumber( + defaultInstance, fieldNumber); + + boolean unknown = false; + boolean packed = false; + if (extension == null) { + unknown = true; // Unknown field. + } else if (wireType == FieldSet.getWireFormatForFieldType( + extension.descriptor.getLiteType(), + false /* isPacked */)) { + packed = false; // Normal, unpacked value. + } else if (extension.descriptor.isRepeated && + extension.descriptor.type.isPackable() && + wireType == FieldSet.getWireFormatForFieldType( + extension.descriptor.getLiteType(), + true /* isPacked */)) { + packed = true; // Packed value. + } else { + unknown = true; // Wrong wire type. + } + + if (unknown) { // Unknown field or wrong wire type. Skip. + return input.skipField(tag); + } + + if (packed) { + int length = input.readRawVarint32(); + int limit = input.pushLimit(length); + if (extension.descriptor.getLiteType() == WireFormat.FieldType.ENUM) { + while (input.getBytesUntilLimit() > 0) { + int rawValue = input.readEnum(); + Object value = + extension.descriptor.getEnumType().findValueByNumber(rawValue); + if (value == null) { + // If the number isn't recognized as a valid value for this + // enum, drop it (don't even add it to unknownFields). + return true; } + extensions.addRepeatedField(extension.descriptor, value); } - input.popLimit(limit); } else { - final Object value; - switch (extension.descriptor.getLiteJavaType()) { - case MESSAGE: { - MessageLite.Builder subBuilder = null; - if (!extension.descriptor.isRepeated()) { - MessageLite existingValue = - (MessageLite) extensions.getField(extension.descriptor); - if (existingValue != null) { - subBuilder = existingValue.toBuilder(); - } - } - if (subBuilder == null) { - subBuilder = extension.messageDefaultInstance.newBuilderForType(); - } - if (extension.descriptor.getLiteType() == - WireFormat.FieldType.GROUP) { - input.readGroup(extension.getNumber(), - subBuilder, extensionRegistry); - } else { - input.readMessage(subBuilder, extensionRegistry); + while (input.getBytesUntilLimit() > 0) { + Object value = + FieldSet.readPrimitiveField(input, + extension.descriptor.getLiteType()); + extensions.addRepeatedField(extension.descriptor, value); + } + } + input.popLimit(limit); + } else { + Object value; + switch (extension.descriptor.getLiteJavaType()) { + case MESSAGE: { + MessageLite.Builder subBuilder = null; + if (!extension.descriptor.isRepeated()) { + MessageLite existingValue = + (MessageLite) extensions.getField(extension.descriptor); + if (existingValue != null) { + subBuilder = existingValue.toBuilder(); } - value = subBuilder.build(); - break; } - case ENUM: - final int rawValue = input.readEnum(); - value = extension.descriptor.getEnumType() - .findValueByNumber(rawValue); - // If the number isn't recognized as a valid value for this enum, - // drop it. - if (value == null) { - return true; - } - break; - default: - value = FieldSet.readPrimitiveField(input, - extension.descriptor.getLiteType()); - break; - } - - if (extension.descriptor.isRepeated()) { - ensureExtensionsIsMutable(); - extensions.addRepeatedField(extension.descriptor, value); - } else { - ensureExtensionsIsMutable(); - extensions.setField(extension.descriptor, value); + if (subBuilder == null) { + subBuilder = extension.messageDefaultInstance.newBuilderForType(); + } + if (extension.descriptor.getLiteType() == + WireFormat.FieldType.GROUP) { + input.readGroup(extension.getNumber(), + subBuilder, extensionRegistry); + } else { + input.readMessage(subBuilder, extensionRegistry); + } + value = subBuilder.build(); + break; } + case ENUM: + int rawValue = input.readEnum(); + value = extension.descriptor.getEnumType() + .findValueByNumber(rawValue); + // If the number isn't recognized as a valid value for this enum, + // drop it. + if (value == null) { + return true; + } + break; + default: + value = FieldSet.readPrimitiveField(input, + extension.descriptor.getLiteType()); + break; } - return true; + if (extension.descriptor.isRepeated()) { + extensions.addRepeatedField(extension.descriptor, value); + } else { + extensions.setField(extension.descriptor, value); + } } - protected final void mergeExtensionFields(final MessageType other) { - ensureExtensionsIsMutable(); - extensions.mergeFrom(((ExtendableMessage) other).extensions); - } + return true; } // ----------------------------------------------------------------- @@ -722,7 +788,7 @@ public abstract class GeneratedMessageLite extends AbstractMessageLite /** * Replaces this object in the output stream with a serialized form. * Part of Java's serialization magic. Generated sub-classes must override - * this method by calling <code>return super.writeReplace();</code> + * this method by calling {@code return super.writeReplace();} * @return a SerializedForm of this message */ protected Object writeReplace() throws ObjectStreamException { diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/Internal.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/Internal.java index 05eab57..81af258 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/Internal.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/Internal.java @@ -103,85 +103,32 @@ public class Internal { * Helper called by generated code to determine if a byte array is a valid * UTF-8 encoded string such that the original bytes can be converted to * a String object and then back to a byte array round tripping the bytes - * without loss. - * <p> - * This is inspired by UTF_8.java in sun.nio.cs. + * without loss. More precisely, returns {@code true} whenever: + * <pre> {@code + * Arrays.equals(byteString.toByteArray(), + * new String(byteString.toByteArray(), "UTF-8").getBytes("UTF-8")) + * }</pre> + * + * <p>This method rejects "overlong" byte sequences, as well as + * 3-byte sequences that would map to a surrogate character, in + * accordance with the restricted definition of UTF-8 introduced in + * Unicode 3.1. Note that the UTF-8 decoder included in Oracle's + * JDK has been modified to also reject "overlong" byte sequences, + * but currently (2011) still accepts 3-byte surrogate character + * byte sequences. + * + * <p>See the Unicode Standard,</br> + * Table 3-6. <em>UTF-8 Bit Distribution</em>,</br> + * Table 3-7. <em>Well Formed UTF-8 Byte Sequences</em>. + * + * <p>As of 2011-02, this method simply returns the result of {@link + * ByteString#isValidUtf8()}. Calling that method directly is preferred. * * @param byteString the string to check * @return whether the byte array is round trippable */ public static boolean isValidUtf8(ByteString byteString) { - int index = 0; - int size = byteString.size(); - // To avoid the masking, we could change this to use bytes; - // Then X > 0xC2 gets turned into X < -0xC2; X < 0x80 - // gets turned into X >= 0, etc. - - while (index < size) { - int byte1 = byteString.byteAt(index++) & 0xFF; - if (byte1 < 0x80) { - // fast loop for single bytes - continue; - - // we know from this point on that we have 2-4 byte forms - } else if (byte1 < 0xC2 || byte1 > 0xF4) { - // catch illegal first bytes: < C2 or > F4 - return false; - } - if (index >= size) { - // fail if we run out of bytes - return false; - } - int byte2 = byteString.byteAt(index++) & 0xFF; - if (byte2 < 0x80 || byte2 > 0xBF) { - // general trail-byte test - return false; - } - if (byte1 <= 0xDF) { - // two-byte form; general trail-byte test is sufficient - continue; - } - - // we know from this point on that we have 3 or 4 byte forms - if (index >= size) { - // fail if we run out of bytes - return false; - } - int byte3 = byteString.byteAt(index++) & 0xFF; - if (byte3 < 0x80 || byte3 > 0xBF) { - // general trail-byte test - return false; - } - if (byte1 <= 0xEF) { - // three-byte form. Vastly more frequent than four-byte forms - // The following has an extra test, but not worth restructuring - if (byte1 == 0xE0 && byte2 < 0xA0 || - byte1 == 0xED && byte2 > 0x9F) { - // check special cases of byte2 - return false; - } - - } else { - // four-byte form - - if (index >= size) { - // fail if we run out of bytes - return false; - } - int byte4 = byteString.byteAt(index++) & 0xFF; - if (byte4 < 0x80 || byte4 > 0xBF) { - // general trail-byte test - return false; - } - // The following has an extra test, but not worth restructuring - if (byte1 == 0xF0 && byte2 < 0x90 || - byte1 == 0xF4 && byte2 > 0x8F) { - // check special cases of byte2 - return false; - } - } - } - return true; + return byteString.isValidUtf8(); } /** diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/InvalidProtocolBufferException.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/InvalidProtocolBufferException.java index 90f7ffb..72d7ff7 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/InvalidProtocolBufferException.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/InvalidProtocolBufferException.java @@ -40,11 +40,32 @@ import java.io.IOException; */ public class InvalidProtocolBufferException extends IOException { private static final long serialVersionUID = -1616151763072450476L; + private MessageLite unfinishedMessage = null; public InvalidProtocolBufferException(final String description) { super(description); } + /** + * Attaches an unfinished message to the exception to support best-effort + * parsing in {@code Parser} interface. + * + * @return this + */ + public InvalidProtocolBufferException setUnfinishedMessage( + MessageLite unfinishedMessage) { + this.unfinishedMessage = unfinishedMessage; + return this; + } + + /** + * Returns the unfinished message attached to the exception, or null if + * no message is attached. + */ + public MessageLite getUnfinishedMessage() { + return unfinishedMessage; + } + static InvalidProtocolBufferException truncatedMessage() { return new InvalidProtocolBufferException( "While parsing a protocol message, the input ended unexpectedly " + diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyField.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyField.java new file mode 100644 index 0000000..df9425e --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyField.java @@ -0,0 +1,216 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.IOException; +import java.util.Iterator; +import java.util.Map.Entry; + +/** + * LazyField encapsulates the logic of lazily parsing message fields. It stores + * the message in a ByteString initially and then parse it on-demand. + * + * LazyField is thread-compatible e.g. concurrent read are safe, however, + * synchronizations are needed under read/write situations. + * + * Now LazyField is only used to lazily load MessageSet. + * TODO(xiangl): Use LazyField to lazily load all messages. + * + * @author xiangl@google.com (Xiang Li) + */ +class LazyField { + + final private MessageLite defaultInstance; + final private ExtensionRegistryLite extensionRegistry; + + // Mutable because it is initialized lazily. + private ByteString bytes; + private volatile MessageLite value; + private volatile boolean isDirty = false; + + public LazyField(MessageLite defaultInstance, + ExtensionRegistryLite extensionRegistry, ByteString bytes) { + this.defaultInstance = defaultInstance; + this.extensionRegistry = extensionRegistry; + this.bytes = bytes; + } + + public MessageLite getValue() { + ensureInitialized(); + return value; + } + + /** + * LazyField is not thread-safe for write access. Synchronizations are needed + * under read/write situations. + */ + public MessageLite setValue(MessageLite value) { + MessageLite originalValue = this.value; + this.value = value; + bytes = null; + isDirty = true; + return originalValue; + } + + /** + * Due to the optional field can be duplicated at the end of serialized + * bytes, which will make the serialized size changed after LazyField + * parsed. Be careful when using this method. + */ + public int getSerializedSize() { + if (isDirty) { + return value.getSerializedSize(); + } + return bytes.size(); + } + + public ByteString toByteString() { + if (!isDirty) { + return bytes; + } + synchronized (this) { + if (!isDirty) { + return bytes; + } + bytes = value.toByteString(); + isDirty = false; + return bytes; + } + } + + @Override + public int hashCode() { + ensureInitialized(); + return value.hashCode(); + } + + @Override + public boolean equals(Object obj) { + ensureInitialized(); + return value.equals(obj); + } + + @Override + public String toString() { + ensureInitialized(); + return value.toString(); + } + + private void ensureInitialized() { + if (value != null) { + return; + } + synchronized (this) { + if (value != null) { + return; + } + try { + if (bytes != null) { + value = defaultInstance.getParserForType() + .parseFrom(bytes, extensionRegistry); + } + } catch (IOException e) { + // TODO(xiangl): Refactory the API to support the exception thrown from + // lazily load messages. + } + } + } + + // ==================================================== + + /** + * LazyEntry and LazyIterator are used to encapsulate the LazyField, when + * users iterate all fields from FieldSet. + */ + static class LazyEntry<K> implements Entry<K, Object> { + private Entry<K, LazyField> entry; + + private LazyEntry(Entry<K, LazyField> entry) { + this.entry = entry; + } + + @Override + public K getKey() { + return entry.getKey(); + } + + @Override + public Object getValue() { + LazyField field = entry.getValue(); + if (field == null) { + return null; + } + return field.getValue(); + } + + public LazyField getField() { + return entry.getValue(); + } + + @Override + public Object setValue(Object value) { + if (!(value instanceof MessageLite)) { + throw new IllegalArgumentException( + "LazyField now only used for MessageSet, " + + "and the value of MessageSet must be an instance of MessageLite"); + } + return entry.getValue().setValue((MessageLite) value); + } + } + + static class LazyIterator<K> implements Iterator<Entry<K, Object>> { + private Iterator<Entry<K, Object>> iterator; + + public LazyIterator(Iterator<Entry<K, Object>> iterator) { + this.iterator = iterator; + } + + @Override + public boolean hasNext() { + return iterator.hasNext(); + } + + @SuppressWarnings("unchecked") + @Override + public Entry<K, Object> next() { + Entry<K, ?> entry = iterator.next(); + if (entry.getValue() instanceof LazyField) { + return new LazyEntry<K>((Entry<K, LazyField>) entry); + } + return (Entry<K, Object>) entry; + } + + @Override + public void remove() { + iterator.remove(); + } + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringArrayList.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringArrayList.java index 1683a640..75c6a4b 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringArrayList.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringArrayList.java @@ -33,8 +33,9 @@ package com.google.protobuf; import java.util.List; import java.util.AbstractList; import java.util.ArrayList; -import java.util.RandomAccess; import java.util.Collection; +import java.util.Collections; +import java.util.RandomAccess; /** * An implementation of {@link LazyStringList} that wraps an ArrayList. Each @@ -72,6 +73,11 @@ public class LazyStringArrayList extends AbstractList<String> list = new ArrayList<Object>(); } + public LazyStringArrayList(LazyStringList from) { + list = new ArrayList<Object>(from.size()); + addAll(from); + } + public LazyStringArrayList(List<String> from) { list = new ArrayList<Object>(from); } @@ -84,7 +90,7 @@ public class LazyStringArrayList extends AbstractList<String> } else { ByteString bs = (ByteString) o; String s = bs.toStringUtf8(); - if (Internal.isValidUtf8(bs)) { + if (bs.isValidUtf8()) { list.set(index, s); } return s; @@ -109,8 +115,21 @@ public class LazyStringArrayList extends AbstractList<String> } @Override + public boolean addAll(Collection<? extends String> c) { + // The default implementation of AbstractCollection.addAll(Collection) + // delegates to add(Object). This implementation instead delegates to + // addAll(int, Collection), which makes a special case for Collections + // which are instances of LazyStringList. + return addAll(size(), c); + } + + @Override public boolean addAll(int index, Collection<? extends String> c) { - boolean ret = list.addAll(index, c); + // When copying from another LazyStringList, directly copy the underlying + // elements rather than forcing each element to be decoded to a String. + Collection<?> collection = c instanceof LazyStringList + ? ((LazyStringList) c).getUnderlyingElements() : c; + boolean ret = list.addAll(index, collection); modCount++; return ret; } @@ -152,4 +171,9 @@ public class LazyStringArrayList extends AbstractList<String> return ((ByteString) o).toStringUtf8(); } } + + @Override + public List<?> getUnderlyingElements() { + return Collections.unmodifiableList(list); + } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringList.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringList.java index 97139ca..630932f 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringList.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/LazyStringList.java @@ -33,7 +33,7 @@ package com.google.protobuf; import java.util.List; /** - * An interface extending List<String> that also provides access to the + * An interface extending {@code List<String>} that also provides access to the * items of the list as UTF8-encoded ByteString objects. This is used by the * protocol buffer implementation to support lazily converting bytes parsed * over the wire to String objects until needed and also increases the @@ -41,9 +41,9 @@ import java.util.List; * ByteString is already cached. * <p> * This only adds additional methods that are required for the use in the - * protocol buffer code in order to be able successfuly round trip byte arrays + * protocol buffer code in order to be able successfully round trip byte arrays * through parsing and serialization without conversion to strings. It's not - * attempting to support the functionality of say List<ByteString>, hence + * attempting to support the functionality of say {@code List<ByteString>}, hence * why only these two very specific methods are added. * * @author jonp@google.com (Jon Perlow) @@ -56,7 +56,7 @@ public interface LazyStringList extends List<String> { * @param index index of the element to return * @return the element at the specified position in this list * @throws IndexOutOfBoundsException if the index is out of range - * (<tt>index < 0 || index >= size()</tt>) + * ({@code index < 0 || index >= size()}) */ ByteString getByteString(int index); @@ -69,4 +69,13 @@ public interface LazyStringList extends List<String> { * is not supported by this list */ void add(ByteString element); + + /** + * Returns an unmodifiable List of the underlying elements, each of + * which is either a {@code String} or its equivalent UTF-8 encoded + * {@code ByteString}. It is an error for the caller to modify the returned + * List, and attempting to do so will result in an + * {@link UnsupportedOperationException}. + */ + List<?> getUnderlyingElements(); } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/LiteralByteString.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/LiteralByteString.java new file mode 100644 index 0000000..93c53dc --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/LiteralByteString.java @@ -0,0 +1,349 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.List; +import java.util.NoSuchElementException; + +/** + * This class implements a {@link com.google.protobuf.ByteString} backed by a + * single array of bytes, contiguous in memory. It supports substring by + * pointing to only a sub-range of the underlying byte array, meaning that a + * substring will reference the full byte-array of the string it's made from, + * exactly as with {@link String}. + * + * @author carlanton@google.com (Carl Haverl) + */ +class LiteralByteString extends ByteString { + + protected final byte[] bytes; + + /** + * Creates a {@code LiteralByteString} backed by the given array, without + * copying. + * + * @param bytes array to wrap + */ + LiteralByteString(byte[] bytes) { + this.bytes = bytes; + } + + @Override + public byte byteAt(int index) { + // Unlike most methods in this class, this one is a direct implementation + // ignoring the potential offset because we need to do range-checking in the + // substring case anyway. + return bytes[index]; + } + + @Override + public int size() { + return bytes.length; + } + + // ================================================================= + // ByteString -> substring + + @Override + public ByteString substring(int beginIndex, int endIndex) { + if (beginIndex < 0) { + throw new IndexOutOfBoundsException( + "Beginning index: " + beginIndex + " < 0"); + } + if (endIndex > size()) { + throw new IndexOutOfBoundsException("End index: " + endIndex + " > " + + size()); + } + int substringLength = endIndex - beginIndex; + if (substringLength < 0) { + throw new IndexOutOfBoundsException( + "Beginning index larger than ending index: " + beginIndex + ", " + + endIndex); + } + + ByteString result; + if (substringLength == 0) { + result = ByteString.EMPTY; + } else { + result = new BoundedByteString(bytes, getOffsetIntoBytes() + beginIndex, + substringLength); + } + return result; + } + + // ================================================================= + // ByteString -> byte[] + + @Override + protected void copyToInternal(byte[] target, int sourceOffset, + int targetOffset, int numberToCopy) { + // Optimized form, not for subclasses, since we don't call + // getOffsetIntoBytes() or check the 'numberToCopy' parameter. + System.arraycopy(bytes, sourceOffset, target, targetOffset, numberToCopy); + } + + @Override + public void copyTo(ByteBuffer target) { + target.put(bytes, getOffsetIntoBytes(), size()); // Copies bytes + } + + @Override + public ByteBuffer asReadOnlyByteBuffer() { + ByteBuffer byteBuffer = + ByteBuffer.wrap(bytes, getOffsetIntoBytes(), size()); + return byteBuffer.asReadOnlyBuffer(); + } + + @Override + public List<ByteBuffer> asReadOnlyByteBufferList() { + // Return the ByteBuffer generated by asReadOnlyByteBuffer() as a singleton + List<ByteBuffer> result = new ArrayList<ByteBuffer>(1); + result.add(asReadOnlyByteBuffer()); + return result; + } + + @Override + public void writeTo(OutputStream outputStream) throws IOException { + outputStream.write(toByteArray()); + } + + @Override + public String toString(String charsetName) + throws UnsupportedEncodingException { + return new String(bytes, getOffsetIntoBytes(), size(), charsetName); + } + + // ================================================================= + // UTF-8 decoding + + @Override + public boolean isValidUtf8() { + int offset = getOffsetIntoBytes(); + return Utf8.isValidUtf8(bytes, offset, offset + size()); + } + + @Override + protected int partialIsValidUtf8(int state, int offset, int length) { + int index = getOffsetIntoBytes() + offset; + return Utf8.partialIsValidUtf8(state, bytes, index, index + length); + } + + // ================================================================= + // equals() and hashCode() + + @Override + public boolean equals(Object other) { + if (other == this) { + return true; + } + if (!(other instanceof ByteString)) { + return false; + } + + if (size() != ((ByteString) other).size()) { + return false; + } + if (size() == 0) { + return true; + } + + if (other instanceof LiteralByteString) { + return equalsRange((LiteralByteString) other, 0, size()); + } else if (other instanceof RopeByteString) { + return other.equals(this); + } else { + throw new IllegalArgumentException( + "Has a new type of ByteString been created? Found " + + other.getClass()); + } + } + + /** + * Check equality of the substring of given length of this object starting at + * zero with another {@code LiteralByteString} substring starting at offset. + * + * @param other what to compare a substring in + * @param offset offset into other + * @param length number of bytes to compare + * @return true for equality of substrings, else false. + */ + boolean equalsRange(LiteralByteString other, int offset, int length) { + if (length > other.size()) { + throw new IllegalArgumentException( + "Length too large: " + length + size()); + } + if (offset + length > other.size()) { + throw new IllegalArgumentException( + "Ran off end of other: " + offset + ", " + length + ", " + + other.size()); + } + + byte[] thisBytes = bytes; + byte[] otherBytes = other.bytes; + int thisLimit = getOffsetIntoBytes() + length; + for (int thisIndex = getOffsetIntoBytes(), otherIndex = + other.getOffsetIntoBytes() + offset; + (thisIndex < thisLimit); ++thisIndex, ++otherIndex) { + if (thisBytes[thisIndex] != otherBytes[otherIndex]) { + return false; + } + } + return true; + } + + /** + * Cached hash value. Intentionally accessed via a data race, which + * is safe because of the Java Memory Model's "no out-of-thin-air values" + * guarantees for ints. + */ + private int hash = 0; + + /** + * Compute the hashCode using the traditional algorithm from {@link + * ByteString}. + * + * @return hashCode value + */ + @Override + public int hashCode() { + int h = hash; + + if (h == 0) { + int size = size(); + h = partialHash(size, 0, size); + if (h == 0) { + h = 1; + } + hash = h; + } + return h; + } + + @Override + protected int peekCachedHashCode() { + return hash; + } + + @Override + protected int partialHash(int h, int offset, int length) { + byte[] thisBytes = bytes; + for (int i = getOffsetIntoBytes() + offset, limit = i + length; i < limit; + i++) { + h = h * 31 + thisBytes[i]; + } + return h; + } + + // ================================================================= + // Input stream + + @Override + public InputStream newInput() { + return new ByteArrayInputStream(bytes, getOffsetIntoBytes(), + size()); // No copy + } + + @Override + public CodedInputStream newCodedInput() { + // We trust CodedInputStream not to modify the bytes, or to give anyone + // else access to them. + return CodedInputStream + .newInstance(bytes, getOffsetIntoBytes(), size()); // No copy + } + + // ================================================================= + // ByteIterator + + @Override + public ByteIterator iterator() { + return new LiteralByteIterator(); + } + + private class LiteralByteIterator implements ByteIterator { + private int position; + private final int limit; + + private LiteralByteIterator() { + position = 0; + limit = size(); + } + + public boolean hasNext() { + return (position < limit); + } + + public Byte next() { + // Boxing calls Byte.valueOf(byte), which does not instantiate. + return nextByte(); + } + + public byte nextByte() { + try { + return bytes[position++]; + } catch (ArrayIndexOutOfBoundsException e) { + throw new NoSuchElementException(e.getMessage()); + } + } + + public void remove() { + throw new UnsupportedOperationException(); + } + } + + // ================================================================= + // Internal methods + + @Override + protected int getTreeDepth() { + return 0; + } + + @Override + protected boolean isBalanced() { + return true; + } + + /** + * Offset into {@code bytes[]} to use, non-zero for substrings. + * + * @return always 0 for this class + */ + protected int getOffsetIntoBytes() { + return 0; + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/Message.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/Message.java index 67c4148..2b88141 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/Message.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/Message.java @@ -50,25 +50,28 @@ import java.util.Map; */ public interface Message extends MessageLite, MessageOrBuilder { + // (From MessageLite, re-declared here only for return type covariance.) + Parser<? extends Message> getParserForType(); + // ----------------------------------------------------------------- // Comparison and hashing /** * Compares the specified object with this message for equality. Returns - * <tt>true</tt> if the given object is a message of the same type (as + * {@code true} if the given object is a message of the same type (as * defined by {@code getDescriptorForType()}) and has identical values for * all of its fields. Subclasses must implement this; inheriting * {@code Object.equals()} is incorrect. * * @param other object to be compared for equality with this message - * @return <tt>true</tt> if the specified object is equal to this message + * @return {@code true} if the specified object is equal to this message */ @Override boolean equals(Object other); /** * Returns the hash code value for this message. The hash code of a message - * should mix the message's type (object identity of the decsriptor) with its + * should mix the message's type (object identity of the descriptor) with its * contents (known and unknown field values). Subclasses must implement this; * inheriting {@code Object.hashCode()} is incorrect. * @@ -83,7 +86,8 @@ public interface Message extends MessageLite, MessageOrBuilder { /** * Converts the message to a string in protocol buffer text format. This is - * just a trivial wrapper around {@link TextFormat#printToString(Message)}. + * just a trivial wrapper around {@link + * TextFormat#printToString(MessageOrBuilder)}. */ @Override String toString(); @@ -145,6 +149,24 @@ public interface Message extends MessageLite, MessageOrBuilder { Builder newBuilderForField(Descriptors.FieldDescriptor field); /** + * Get a nested builder instance for the given field. + * <p> + * Normally, we hold a reference to the immutable message object for the + * message type field. Some implementations(the generated message builders), + * however, can also hold a reference to the builder object (a nested + * builder) for the field. + * <p> + * If the field is already backed up by a nested builder, the nested builder + * will be returned. Otherwise, a new field builder will be created and + * returned. The original message field (if exist) will be merged into the + * field builder, which will then be nested into its parent builder. + * <p> + * NOTE: implementations that do not support nested builders will throw + * <code>UnsupportedException</code>. + */ + Builder getFieldBuilder(Descriptors.FieldDescriptor field); + + /** * Sets a field to the given value. The value must be of the correct type * for this field, i.e. the same type that * {@link Message#getField(Descriptors.FieldDescriptor)} would return. diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLite.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLite.java index 31b8256..e5b9a47 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLite.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLite.java @@ -79,6 +79,12 @@ public interface MessageLite extends MessageLiteOrBuilder { */ int getSerializedSize(); + + /** + * Gets the parser for a message of the same type as this message. + */ + Parser<? extends MessageLite> getParserForType(); + // ----------------------------------------------------------------- // Convenience methods. @@ -144,11 +150,8 @@ public interface MessageLite extends MessageLiteOrBuilder { Builder clear(); /** - * Construct the final message. Once this is called, the Builder is no - * longer valid, and calling any other method will result in undefined - * behavior and may throw a NullPointerException. If you need to continue - * working with the builder after calling {@code build()}, {@code clone()} - * it first. + * Constructs the message based on the state of the Builder. Subsequent + * changes to the Builder will not affect the returned message. * @throws UninitializedMessageException The message is missing one or more * required fields (i.e. {@link #isInitialized()} returns false). * Use {@link #buildPartial()} to bypass this check. @@ -158,11 +161,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Like {@link #build()}, but does not throw an exception if the message * is missing required fields. Instead, a partial message is returned. - * Once this is called, the Builder is no longer valid, and calling any - * will result in undefined behavior and may throw a NullPointerException. - * - * If you need to continue working with the builder after calling - * {@code buildPartial()}, {@code clone()} it first. + * Subsequent changes to the Builder will not affect the returned message. */ MessageLite buildPartial(); @@ -174,7 +173,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Parses a message of this type from the input and merges it with this - * message, as if using {@link Builder#mergeFrom(MessageLite)}. + * message. * * <p>Warning: This does not verify that all required fields are present in * the input message. If you call {@link #build()} without setting all @@ -184,11 +183,6 @@ public interface MessageLite extends MessageLiteOrBuilder { * <ul> * <li>Call {@link #isInitialized()} to verify that all required fields * are set before building. - * <li>Parse the message separately using one of the static - * {@code parseFrom} methods, then use {@link #mergeFrom(MessageLite)} - * to merge it with this one. {@code parseFrom} will throw an - * {@link InvalidProtocolBufferException} (an {@code IOException}) - * if some required fields are missing. * <li>Use {@code buildPartial()} to build, which ignores missing * required fields. * </ul> @@ -225,7 +219,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Parse {@code data} as a message of this type and merge it with the * message being built. This is just a small wrapper around - * {@link #mergeFrom(CodedInputStream,ExtensionRegistry)}. + * {@link #mergeFrom(CodedInputStream,ExtensionRegistryLite)}. * * @return this */ @@ -255,7 +249,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Parse {@code data} as a message of this type and merge it with the * message being built. This is just a small wrapper around - * {@link #mergeFrom(CodedInputStream,ExtensionRegistry)}. + * {@link #mergeFrom(CodedInputStream,ExtensionRegistryLite)}. * * @return this */ @@ -266,7 +260,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Parse {@code data} as a message of this type and merge it with the * message being built. This is just a small wrapper around - * {@link #mergeFrom(CodedInputStream,ExtensionRegistry)}. + * {@link #mergeFrom(CodedInputStream,ExtensionRegistryLite)}. * * @return this */ @@ -293,7 +287,7 @@ public interface MessageLite extends MessageLiteOrBuilder { /** * Parse a message of this type from {@code input} and merge it with the * message being built. This is just a small wrapper around - * {@link #mergeFrom(CodedInputStream,ExtensionRegistry)}. + * {@link #mergeFrom(CodedInputStream,ExtensionRegistryLite)}. * * @return this */ @@ -308,9 +302,9 @@ public interface MessageLite extends MessageLiteOrBuilder { * {@link MessageLite#writeDelimitedTo(OutputStream)} to write messages in * this format. * - * @returns True if successful, or false if the stream is at EOF when the - * method starts. Any other error (including reaching EOF during - * parsing) will cause an exception to be thrown. + * @return True if successful, or false if the stream is at EOF when the + * method starts. Any other error (including reaching EOF during + * parsing) will cause an exception to be thrown. */ boolean mergeDelimitedFrom(InputStream input) throws IOException; diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLiteOrBuilder.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLiteOrBuilder.java index 7cc72e9..05b2b16 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLiteOrBuilder.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageLiteOrBuilder.java @@ -52,6 +52,8 @@ public interface MessageLiteOrBuilder { /** * Returns true if all required fields in the message and all embedded * messages are set, false otherwise. + * + * <p>See also: {@link MessageOrBuilder#getInitializationErrorString()} */ boolean isInitialized(); diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageOrBuilder.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageOrBuilder.java index 0132e7c..bf62d45 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageOrBuilder.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/MessageOrBuilder.java @@ -30,6 +30,7 @@ package com.google.protobuf; +import java.util.List; import java.util.Map; /** @@ -45,6 +46,24 @@ public interface MessageOrBuilder extends MessageLiteOrBuilder { Message getDefaultInstanceForType(); /** + * Returns a list of field paths (e.g. "foo.bar.baz") of required fields + * which are not set in this message. You should call + * {@link MessageLiteOrBuilder#isInitialized()} first to check if there + * are any missing fields, as that method is likely to be much faster + * than this one even when the message is fully-initialized. + */ + List<String> findInitializationErrors(); + + /** + * Returns a comma-delimited list of required fields which are not set + * in this message object. You should call + * {@link MessageLiteOrBuilder#isInitialized()} first to check if there + * are any missing fields, as that method is likely to be much faster + * than this one even when the message is fully-initialized. + */ + String getInitializationErrorString(); + + /** * Get the message's type's descriptor. This differs from the * {@code getDescriptor()} method of generated message classes in that * this method is an abstract method of the {@code Message} interface @@ -80,7 +99,7 @@ public interface MessageOrBuilder extends MessageLiteOrBuilder { /** * Obtains the value of the given field, or the default value if it is * not set. For primitive fields, the boxed primitive value is returned. - * For enum fields, the EnumValueDescriptor for the value is returend. For + * For enum fields, the EnumValueDescriptor for the value is returned. For * embedded message fields, the sub-message is returned. For repeated * fields, a java.util.List is returned. */ @@ -98,7 +117,7 @@ public interface MessageOrBuilder extends MessageLiteOrBuilder { /** * Gets an element of a repeated field. For primitive fields, the boxed * primitive value is returned. For enum fields, the EnumValueDescriptor - * for the value is returend. For embedded message fields, the sub-message + * for the value is returned. For embedded message fields, the sub-message * is returned. * @throws IllegalArgumentException The field is not a repeated field, or * {@code field.getContainingType() != getDescriptorForType()}. diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/Parser.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/Parser.java new file mode 100644 index 0000000..7d8e821 --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/Parser.java @@ -0,0 +1,259 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.InputStream; + +/** + * Abstract interface for parsing Protocol Messages. + * + * @author liujisi@google.com (Pherl Liu) + */ +public interface Parser<MessageType> { + /** + * Parses a message of {@code MessageType} from the input. + * + * <p>Note: The caller should call + * {@link CodedInputStream#checkLastTagWas(int)} after calling this to + * verify that the last tag seen was the appropriate end-group tag, + * or zero for EOF. + */ + public MessageType parseFrom(CodedInputStream input) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(CodedInputStream)}, but also parses extensions. + * The extensions that you want to be able to parse must be registered in + * {@code extensionRegistry}. Extensions not in the registry will be treated + * as unknown fields. + */ + public MessageType parseFrom(CodedInputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(CodedInputStream)}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialFrom(CodedInputStream input) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(CodedInputStream input, ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialFrom(CodedInputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + // --------------------------------------------------------------- + // Convenience methods. + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around {@link #parseFrom(CodedInputStream)}. + */ + public MessageType parseFrom(ByteString data) + throws InvalidProtocolBufferException; + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around + * {@link #parseFrom(CodedInputStream, ExtensionRegistryLite)}. + */ + public MessageType parseFrom(ByteString data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(ByteString)}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialFrom(ByteString data) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(ByteString, ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialFrom(ByteString data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around {@link #parseFrom(CodedInputStream)}. + */ + public MessageType parseFrom(byte[] data, int off, int len) + throws InvalidProtocolBufferException; + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around + * {@link #parseFrom(CodedInputStream, ExtensionRegistryLite)}. + */ + public MessageType parseFrom(byte[] data, int off, int len, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around {@link #parseFrom(CodedInputStream)}. + */ + public MessageType parseFrom(byte[] data) + throws InvalidProtocolBufferException; + + /** + * Parses {@code data} as a message of {@code MessageType}. + * This is just a small wrapper around + * {@link #parseFrom(CodedInputStream, ExtensionRegistryLite)}. + */ + public MessageType parseFrom(byte[] data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(byte[], int, int)}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialFrom(byte[] data, int off, int len) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(ByteString, ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialFrom(byte[] data, int off, int len, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(byte[])}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialFrom(byte[] data) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(byte[], ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialFrom(byte[] data, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Parse a message of {@code MessageType} from {@code input}. + * This is just a small wrapper around {@link #parseFrom(CodedInputStream)}. + * Note that this method always reads the <i>entire</i> input (unless it + * throws an exception). If you want it to stop earlier, you will need to + * wrap your input in some wrapper stream that limits reading. Or, use + * {@link MessageLite#writeDelimitedTo(java.io.OutputStream)} to write your + * message and {@link #parseDelimitedFrom(InputStream)} to read it. + * <p> + * Despite usually reading the entire input, this does not close the stream. + */ + public MessageType parseFrom(InputStream input) + throws InvalidProtocolBufferException; + + /** + * Parses a message of {@code MessageType} from {@code input}. + * This is just a small wrapper around + * {@link #parseFrom(CodedInputStream, ExtensionRegistryLite)}. + */ + public MessageType parseFrom(InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(InputStream)}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialFrom(InputStream input) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(InputStream, ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialFrom(InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseFrom(InputStream)}, but does not read util EOF. + * Instead, the size of message (encoded as a varint) is read first, + * then the message data. Use + * {@link MessageLite#writeDelimitedTo(java.io.OutputStream)} to write + * messages in this format. + * + * @return True if successful, or false if the stream is at EOF when the + * method starts. Any other error (including reaching EOF during + * parsing) will cause an exception to be thrown. + */ + public MessageType parseDelimitedFrom(InputStream input) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseDelimitedFrom(InputStream)} but supporting extensions. + */ + public MessageType parseDelimitedFrom(InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseDelimitedFrom(InputStream)}, but does not throw an + * exception if the message is missing required fields. Instead, a partial + * message is returned. + */ + public MessageType parsePartialDelimitedFrom(InputStream input) + throws InvalidProtocolBufferException; + + /** + * Like {@link #parseDelimitedFrom(InputStream, ExtensionRegistryLite)}, + * but does not throw an exception if the message is missing required fields. + * Instead, a partial message is returned. + */ + public MessageType parsePartialDelimitedFrom( + InputStream input, + ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException; +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/RepeatedFieldBuilder.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/RepeatedFieldBuilder.java index 0024f79..65d9270 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/RepeatedFieldBuilder.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/RepeatedFieldBuilder.java @@ -37,22 +37,22 @@ import java.util.Collections; import java.util.List; /** - * <code>RepeatedFieldBuilder</code> implements a structure that a protocol + * {@code RepeatedFieldBuilder} implements a structure that a protocol * message uses to hold a repeated field of other protocol messages. It supports * the classical use case of adding immutable {@link Message}'s to the * repeated field and is highly optimized around this (no extra memory * allocations and sharing of immutable arrays). * <br> * It also supports the additional use case of adding a {@link Message.Builder} - * to the repeated field and deferring conversion of that <code>Builder</code> - * to an immutable <code>Message</code>. In this way, it's possible to maintain - * a tree of <code>Builder</code>'s that acts as a fully read/write data + * to the repeated field and deferring conversion of that {@code Builder} + * to an immutable {@code Message}. In this way, it's possible to maintain + * a tree of {@code Builder}'s that acts as a fully read/write data * structure. * <br> * Logically, one can think of a tree of builders as converting the entire tree * to messages when build is called on the root or when any method is called * that desires a Message instead of a Builder. In terms of the implementation, - * the <code>SingleFieldBuilder</code> and <code>RepeatedFieldBuilder</code> + * the {@code SingleFieldBuilder} and {@code RepeatedFieldBuilder} * classes cache messages that were created so that messages only need to be * created when some change occured in its builder or a builder for one of its * descendants. @@ -192,7 +192,7 @@ public class RepeatedFieldBuilder /** * Get the message at the specified index. If the message is currently stored - * as a <code>Builder</code>, it is converted to a <code>Message</code> by + * as a {@code Builder}, it is converted to a {@code Message} by * calling {@link Message.Builder#buildPartial} on it. * * @param index the index of the message to get @@ -204,7 +204,7 @@ public class RepeatedFieldBuilder /** * Get the message at the specified index. If the message is currently stored - * as a <code>Builder</code>, it is converted to a <code>Message</code> by + * as a {@code Builder}, it is converted to a {@code Message} by * calling {@link Message.Builder#buildPartial} on it. * * @param index the index of the message to get diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/RopeByteString.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/RopeByteString.java new file mode 100644 index 0000000..8d44d11 --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/RopeByteString.java @@ -0,0 +1,945 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; +import java.io.ByteArrayInputStream; +import java.nio.ByteBuffer; +import java.util.ArrayDeque; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Deque; +import java.util.Iterator; +import java.util.List; +import java.util.NoSuchElementException; + +/** + * Class to represent {@code ByteStrings} formed by concatenation of other + * ByteStrings, without copying the data in the pieces. The concatenation is + * represented as a tree whose leaf nodes are each a {@link LiteralByteString}. + * + * <p>Most of the operation here is inspired by the now-famous paper <a + * href="http://www.cs.ubc.ca/local/reading/proceedings/spe91-95/spe/vol25/issue12/spe986.pdf"> + * BAP95 </a> Ropes: an Alternative to Strings hans-j. boehm, russ atkinson and + * michael plass + * + * <p>The algorithms described in the paper have been implemented for character + * strings in {@link com.google.common.string.Rope} and in the c++ class {@code + * cord.cc}. + * + * <p>Fundamentally the Rope algorithm represents the collection of pieces as a + * binary tree. BAP95 uses a Fibonacci bound relating depth to a minimum + * sequence length, sequences that are too short relative to their depth cause a + * tree rebalance. More precisely, a tree of depth d is "balanced" in the + * terminology of BAP95 if its length is at least F(d+2), where F(n) is the + * n-the Fibonacci number. Thus for depths 0, 1, 2, 3, 4, 5,... we have minimum + * lengths 1, 2, 3, 5, 8, 13,... + * + * @author carlanton@google.com (Carl Haverl) + */ +class RopeByteString extends ByteString { + + /** + * BAP95. Let Fn be the nth Fibonacci number. A {@link RopeByteString} of + * depth n is "balanced", i.e flat enough, if its length is at least Fn+2, + * e.g. a "balanced" {@link RopeByteString} of depth 1 must have length at + * least 2, of depth 4 must have length >= 8, etc. + * + * <p>There's nothing special about using the Fibonacci numbers for this, but + * they are a reasonable sequence for encapsulating the idea that we are OK + * with longer strings being encoded in deeper binary trees. + * + * <p>For 32-bit integers, this array has length 46. + */ + private static final int[] minLengthByDepth; + + static { + // Dynamically generate the list of Fibonacci numbers the first time this + // class is accessed. + List<Integer> numbers = new ArrayList<Integer>(); + + // we skip the first Fibonacci number (1). So instead of: 1 1 2 3 5 8 ... + // we have: 1 2 3 5 8 ... + int f1 = 1; + int f2 = 1; + + // get all the values until we roll over. + while (f2 > 0) { + numbers.add(f2); + int temp = f1 + f2; + f1 = f2; + f2 = temp; + } + + // we include this here so that we can index this array to [x + 1] in the + // loops below. + numbers.add(Integer.MAX_VALUE); + minLengthByDepth = new int[numbers.size()]; + for (int i = 0; i < minLengthByDepth.length; i++) { + // unbox all the values + minLengthByDepth[i] = numbers.get(i); + } + } + + private final int totalLength; + private final ByteString left; + private final ByteString right; + private final int leftLength; + private final int treeDepth; + + /** + * Create a new RopeByteString, which can be thought of as a new tree node, by + * recording references to the two given strings. + * + * @param left string on the left of this node, should have {@code size() > + * 0} + * @param right string on the right of this node, should have {@code size() > + * 0} + */ + private RopeByteString(ByteString left, ByteString right) { + this.left = left; + this.right = right; + leftLength = left.size(); + totalLength = leftLength + right.size(); + treeDepth = Math.max(left.getTreeDepth(), right.getTreeDepth()) + 1; + } + + /** + * Concatenate the given strings while performing various optimizations to + * slow the growth rate of tree depth and tree node count. The result is + * either a {@link LiteralByteString} or a {@link RopeByteString} + * depending on which optimizations, if any, were applied. + * + * <p>Small pieces of length less than {@link + * ByteString#CONCATENATE_BY_COPY_SIZE} may be copied by value here, as in + * BAP95. Large pieces are referenced without copy. + * + * @param left string on the left + * @param right string on the right + * @return concatenation representing the same sequence as the given strings + */ + static ByteString concatenate(ByteString left, ByteString right) { + ByteString result; + RopeByteString leftRope = + (left instanceof RopeByteString) ? (RopeByteString) left : null; + if (right.size() == 0) { + result = left; + } else if (left.size() == 0) { + result = right; + } else { + int newLength = left.size() + right.size(); + if (newLength < ByteString.CONCATENATE_BY_COPY_SIZE) { + // Optimization from BAP95: For short (leaves in paper, but just short + // here) total length, do a copy of data to a new leaf. + result = concatenateBytes(left, right); + } else if (leftRope != null + && leftRope.right.size() + right.size() < CONCATENATE_BY_COPY_SIZE) { + // Optimization from BAP95: As an optimization of the case where the + // ByteString is constructed by repeated concatenate, recognize the case + // where a short string is concatenated to a left-hand node whose + // right-hand branch is short. In the paper this applies to leaves, but + // we just look at the length here. This has the advantage of shedding + // references to unneeded data when substrings have been taken. + // + // When we recognize this case, we do a copy of the data and create a + // new parent node so that the depth of the result is the same as the + // given left tree. + ByteString newRight = concatenateBytes(leftRope.right, right); + result = new RopeByteString(leftRope.left, newRight); + } else if (leftRope != null + && leftRope.left.getTreeDepth() > leftRope.right.getTreeDepth() + && leftRope.getTreeDepth() > right.getTreeDepth()) { + // Typically for concatenate-built strings the left-side is deeper than + // the right. This is our final attempt to concatenate without + // increasing the tree depth. We'll redo the the node on the RHS. This + // is yet another optimization for building the string by repeatedly + // concatenating on the right. + ByteString newRight = new RopeByteString(leftRope.right, right); + result = new RopeByteString(leftRope.left, newRight); + } else { + // Fine, we'll add a node and increase the tree depth--unless we + // rebalance ;^) + int newDepth = Math.max(left.getTreeDepth(), right.getTreeDepth()) + 1; + if (newLength >= minLengthByDepth[newDepth]) { + // The tree is shallow enough, so don't rebalance + result = new RopeByteString(left, right); + } else { + result = new Balancer().balance(left, right); + } + } + } + return result; + } + + /** + * Concatenates two strings by copying data values. This is called in a few + * cases in order to reduce the growth of the number of tree nodes. + * + * @param left string on the left + * @param right string on the right + * @return string formed by copying data bytes + */ + private static LiteralByteString concatenateBytes(ByteString left, + ByteString right) { + int leftSize = left.size(); + int rightSize = right.size(); + byte[] bytes = new byte[leftSize + rightSize]; + left.copyTo(bytes, 0, 0, leftSize); + right.copyTo(bytes, 0, leftSize, rightSize); + return new LiteralByteString(bytes); // Constructor wraps bytes + } + + /** + * Create a new RopeByteString for testing only while bypassing all the + * defenses of {@link #concatenate(ByteString, ByteString)}. This allows + * testing trees of specific structure. We are also able to insert empty + * leaves, though these are dis-allowed, so that we can make sure the + * implementation can withstand their presence. + * + * @param left string on the left of this node + * @param right string on the right of this node + * @return an unsafe instance for testing only + */ + static RopeByteString newInstanceForTest(ByteString left, ByteString right) { + return new RopeByteString(left, right); + } + + /** + * Gets the byte at the given index. + * Throws {@link ArrayIndexOutOfBoundsException} for backwards-compatibility + * reasons although it would more properly be {@link + * IndexOutOfBoundsException}. + * + * @param index index of byte + * @return the value + * @throws ArrayIndexOutOfBoundsException {@code index} is < 0 or >= size + */ + @Override + public byte byteAt(int index) { + if (index < 0) { + throw new ArrayIndexOutOfBoundsException("Index < 0: " + index); + } + if (index > totalLength) { + throw new ArrayIndexOutOfBoundsException( + "Index > length: " + index + ", " + totalLength); + } + + byte result; + // Find the relevant piece by recursive descent + if (index < leftLength) { + result = left.byteAt(index); + } else { + result = right.byteAt(index - leftLength); + } + return result; + } + + @Override + public int size() { + return totalLength; + } + + // ================================================================= + // Pieces + + @Override + protected int getTreeDepth() { + return treeDepth; + } + + /** + * Determines if the tree is balanced according to BAP95, which means the tree + * is flat-enough with respect to the bounds. Note that this definition of + * balanced is one where sub-trees of balanced trees are not necessarily + * balanced. + * + * @return true if the tree is balanced + */ + @Override + protected boolean isBalanced() { + return totalLength >= minLengthByDepth[treeDepth]; + } + + /** + * Takes a substring of this one. This involves recursive descent along the + * left and right edges of the substring, and referencing any wholly contained + * segments in between. Any leaf nodes entirely uninvolved in the substring + * will not be referenced by the substring. + * + * <p>Substrings of {@code length < 2} should result in at most a single + * recursive call chain, terminating at a leaf node. Thus the result will be a + * {@link LiteralByteString}. {@link #RopeByteString(ByteString, + * ByteString)}. + * + * @param beginIndex start at this index + * @param endIndex the last character is the one before this index + * @return substring leaf node or tree + */ + @Override + public ByteString substring(int beginIndex, int endIndex) { + if (beginIndex < 0) { + throw new IndexOutOfBoundsException( + "Beginning index: " + beginIndex + " < 0"); + } + if (endIndex > totalLength) { + throw new IndexOutOfBoundsException( + "End index: " + endIndex + " > " + totalLength); + } + int substringLength = endIndex - beginIndex; + if (substringLength < 0) { + throw new IndexOutOfBoundsException( + "Beginning index larger than ending index: " + beginIndex + ", " + + endIndex); + } + + ByteString result; + if (substringLength == 0) { + // Empty substring + result = ByteString.EMPTY; + } else if (substringLength == totalLength) { + // The whole string + result = this; + } else { + // Proper substring + if (endIndex <= leftLength) { + // Substring on the left + result = left.substring(beginIndex, endIndex); + } else if (beginIndex >= leftLength) { + // Substring on the right + result = right + .substring(beginIndex - leftLength, endIndex - leftLength); + } else { + // Split substring + ByteString leftSub = left.substring(beginIndex); + ByteString rightSub = right.substring(0, endIndex - leftLength); + // Intentionally not rebalancing, since in many cases these two + // substrings will already be less deep than the top-level + // RopeByteString we're taking a substring of. + result = new RopeByteString(leftSub, rightSub); + } + } + return result; + } + + // ================================================================= + // ByteString -> byte[] + + @Override + protected void copyToInternal(byte[] target, int sourceOffset, + int targetOffset, int numberToCopy) { + if (sourceOffset + numberToCopy <= leftLength) { + left.copyToInternal(target, sourceOffset, targetOffset, numberToCopy); + } else if (sourceOffset >= leftLength) { + right.copyToInternal(target, sourceOffset - leftLength, targetOffset, + numberToCopy); + } else { + int leftLength = this.leftLength - sourceOffset; + left.copyToInternal(target, sourceOffset, targetOffset, leftLength); + right.copyToInternal(target, 0, targetOffset + leftLength, + numberToCopy - leftLength); + } + } + + @Override + public void copyTo(ByteBuffer target) { + left.copyTo(target); + right.copyTo(target); + } + + @Override + public ByteBuffer asReadOnlyByteBuffer() { + ByteBuffer byteBuffer = ByteBuffer.wrap(toByteArray()); + return byteBuffer.asReadOnlyBuffer(); + } + + @Override + public List<ByteBuffer> asReadOnlyByteBufferList() { + // Walk through the list of LiteralByteString's that make up this + // rope, and add each one as a read-only ByteBuffer. + List<ByteBuffer> result = new ArrayList<ByteBuffer>(); + PieceIterator pieces = new PieceIterator(this); + while (pieces.hasNext()) { + LiteralByteString byteString = pieces.next(); + result.add(byteString.asReadOnlyByteBuffer()); + } + return result; + } + + @Override + public void writeTo(OutputStream outputStream) throws IOException { + left.writeTo(outputStream); + right.writeTo(outputStream); + } + + @Override + public String toString(String charsetName) + throws UnsupportedEncodingException { + return new String(toByteArray(), charsetName); + } + + // ================================================================= + // UTF-8 decoding + + @Override + public boolean isValidUtf8() { + int leftPartial = left.partialIsValidUtf8(Utf8.COMPLETE, 0, leftLength); + int state = right.partialIsValidUtf8(leftPartial, 0, right.size()); + return state == Utf8.COMPLETE; + } + + @Override + protected int partialIsValidUtf8(int state, int offset, int length) { + int toIndex = offset + length; + if (toIndex <= leftLength) { + return left.partialIsValidUtf8(state, offset, length); + } else if (offset >= leftLength) { + return right.partialIsValidUtf8(state, offset - leftLength, length); + } else { + int leftLength = this.leftLength - offset; + int leftPartial = left.partialIsValidUtf8(state, offset, leftLength); + return right.partialIsValidUtf8(leftPartial, 0, length - leftLength); + } + } + + // ================================================================= + // equals() and hashCode() + + @Override + public boolean equals(Object other) { + if (other == this) { + return true; + } + if (!(other instanceof ByteString)) { + return false; + } + + ByteString otherByteString = (ByteString) other; + if (totalLength != otherByteString.size()) { + return false; + } + if (totalLength == 0) { + return true; + } + + // You don't really want to be calling equals on long strings, but since + // we cache the hashCode, we effectively cache inequality. We use the cached + // hashCode if it's already computed. It's arguable we should compute the + // hashCode here, and if we're going to be testing a bunch of byteStrings, + // it might even make sense. + if (hash != 0) { + int cachedOtherHash = otherByteString.peekCachedHashCode(); + if (cachedOtherHash != 0 && hash != cachedOtherHash) { + return false; + } + } + + return equalsFragments(otherByteString); + } + + /** + * Determines if this string is equal to another of the same length by + * iterating over the leaf nodes. On each step of the iteration, the + * overlapping segments of the leaves are compared. + * + * @param other string of the same length as this one + * @return true if the values of this string equals the value of the given + * one + */ + private boolean equalsFragments(ByteString other) { + int thisOffset = 0; + Iterator<LiteralByteString> thisIter = new PieceIterator(this); + LiteralByteString thisString = thisIter.next(); + + int thatOffset = 0; + Iterator<LiteralByteString> thatIter = new PieceIterator(other); + LiteralByteString thatString = thatIter.next(); + + int pos = 0; + while (true) { + int thisRemaining = thisString.size() - thisOffset; + int thatRemaining = thatString.size() - thatOffset; + int bytesToCompare = Math.min(thisRemaining, thatRemaining); + + // At least one of the offsets will be zero + boolean stillEqual = (thisOffset == 0) + ? thisString.equalsRange(thatString, thatOffset, bytesToCompare) + : thatString.equalsRange(thisString, thisOffset, bytesToCompare); + if (!stillEqual) { + return false; + } + + pos += bytesToCompare; + if (pos >= totalLength) { + if (pos == totalLength) { + return true; + } + throw new IllegalStateException(); + } + // We always get to the end of at least one of the pieces + if (bytesToCompare == thisRemaining) { // If reached end of this + thisOffset = 0; + thisString = thisIter.next(); + } else { + thisOffset += bytesToCompare; + } + if (bytesToCompare == thatRemaining) { // If reached end of that + thatOffset = 0; + thatString = thatIter.next(); + } else { + thatOffset += bytesToCompare; + } + } + } + + /** + * Cached hash value. Intentionally accessed via a data race, which is safe + * because of the Java Memory Model's "no out-of-thin-air values" guarantees + * for ints. + */ + private int hash = 0; + + @Override + public int hashCode() { + int h = hash; + + if (h == 0) { + h = totalLength; + h = partialHash(h, 0, totalLength); + if (h == 0) { + h = 1; + } + hash = h; + } + return h; + } + + @Override + protected int peekCachedHashCode() { + return hash; + } + + @Override + protected int partialHash(int h, int offset, int length) { + int toIndex = offset + length; + if (toIndex <= leftLength) { + return left.partialHash(h, offset, length); + } else if (offset >= leftLength) { + return right.partialHash(h, offset - leftLength, length); + } else { + int leftLength = this.leftLength - offset; + int leftPartial = left.partialHash(h, offset, leftLength); + return right.partialHash(leftPartial, 0, length - leftLength); + } + } + + // ================================================================= + // Input stream + + @Override + public CodedInputStream newCodedInput() { + return CodedInputStream.newInstance(new RopeInputStream()); + } + + @Override + public InputStream newInput() { + return new RopeInputStream(); + } + + /** + * This class implements the balancing algorithm of BAP95. In the paper the + * authors use an array to keep track of pieces, while here we use a stack. + * The tree is balanced by traversing subtrees in left to right order, and the + * stack always contains the part of the string we've traversed so far. + * + * <p>One surprising aspect of the algorithm is the result of balancing is not + * necessarily balanced, though it is nearly balanced. For details, see + * BAP95. + */ + private static class Balancer { + // Stack containing the part of the string, starting from the left, that + // we've already traversed. The final string should be the equivalent of + // concatenating the strings on the stack from bottom to top. + private final Deque<ByteString> prefixesStack = + new ArrayDeque<ByteString>(minLengthByDepth.length); + + private ByteString balance(ByteString left, ByteString right) { + doBalance(left); + doBalance(right); + + // Sweep stack to gather the result + ByteString partialString = prefixesStack.pop(); + while (!prefixesStack.isEmpty()) { + ByteString newLeft = prefixesStack.pop(); + partialString = new RopeByteString(newLeft, partialString); + } + // We should end up with a RopeByteString since at a minimum we will + // create one from concatenating left and right + return partialString; + } + + private void doBalance(ByteString root) { + // BAP95: Insert balanced subtrees whole. This means the result might not + // be balanced, leading to repeated rebalancings on concatenate. However, + // these rebalancings are shallow due to ignoring balanced subtrees, and + // relatively few calls to insert() result. + if (root.isBalanced()) { + insert(root); + } else if (root instanceof RopeByteString) { + RopeByteString rbs = (RopeByteString) root; + doBalance(rbs.left); + doBalance(rbs.right); + } else { + throw new IllegalArgumentException( + "Has a new type of ByteString been created? Found " + + root.getClass()); + } + } + + /** + * Push a string on the balance stack (BAP95). BAP95 uses an array and + * calls the elements in the array 'bins'. We instead use a stack, so the + * 'bins' of lengths are represented by differences between the elements of + * minLengthByDepth. + * + * <p>If the length bin for our string, and all shorter length bins, are + * empty, we just push it on the stack. Otherwise, we need to start + * concatenating, putting the given string in the "middle" and continuing + * until we land in an empty length bin that matches the length of our + * concatenation. + * + * @param byteString string to place on the balance stack + */ + private void insert(ByteString byteString) { + int depthBin = getDepthBinForLength(byteString.size()); + int binEnd = minLengthByDepth[depthBin + 1]; + + // BAP95: Concatenate all trees occupying bins representing the length of + // our new piece or of shorter pieces, to the extent that is possible. + // The goal is to clear the bin which our piece belongs in, but that may + // not be entirely possible if there aren't enough longer bins occupied. + if (prefixesStack.isEmpty() || prefixesStack.peek().size() >= binEnd) { + prefixesStack.push(byteString); + } else { + int binStart = minLengthByDepth[depthBin]; + + // Concatenate the subtrees of shorter length + ByteString newTree = prefixesStack.pop(); + while (!prefixesStack.isEmpty() + && prefixesStack.peek().size() < binStart) { + ByteString left = prefixesStack.pop(); + newTree = new RopeByteString(left, newTree); + } + + // Concatenate the given string + newTree = new RopeByteString(newTree, byteString); + + // Continue concatenating until we land in an empty bin + while (!prefixesStack.isEmpty()) { + depthBin = getDepthBinForLength(newTree.size()); + binEnd = minLengthByDepth[depthBin + 1]; + if (prefixesStack.peek().size() < binEnd) { + ByteString left = prefixesStack.pop(); + newTree = new RopeByteString(left, newTree); + } else { + break; + } + } + prefixesStack.push(newTree); + } + } + + private int getDepthBinForLength(int length) { + int depth = Arrays.binarySearch(minLengthByDepth, length); + if (depth < 0) { + // It wasn't an exact match, so convert to the index of the containing + // fragment, which is one less even than the insertion point. + int insertionPoint = -(depth + 1); + depth = insertionPoint - 1; + } + + return depth; + } + } + + /** + * This class is a continuable tree traversal, which keeps the state + * information which would exist on the stack in a recursive traversal instead + * on a stack of "Bread Crumbs". The maximum depth of the stack in this + * iterator is the same as the depth of the tree being traversed. + * + * <p>This iterator is used to implement + * {@link RopeByteString#equalsFragments(ByteString)}. + */ + private static class PieceIterator implements Iterator<LiteralByteString> { + + private final Deque<RopeByteString> breadCrumbs = + new ArrayDeque<RopeByteString>(minLengthByDepth.length); + private LiteralByteString next; + + private PieceIterator(ByteString root) { + next = getLeafByLeft(root); + } + + private LiteralByteString getLeafByLeft(ByteString root) { + ByteString pos = root; + while (pos instanceof RopeByteString) { + RopeByteString rbs = (RopeByteString) pos; + breadCrumbs.push(rbs); + pos = rbs.left; + } + return (LiteralByteString) pos; + } + + private LiteralByteString getNextNonEmptyLeaf() { + while (true) { + // Almost always, we go through this loop exactly once. However, if + // we discover an empty string in the rope, we toss it and try again. + if (breadCrumbs.isEmpty()) { + return null; + } else { + LiteralByteString result = getLeafByLeft(breadCrumbs.pop().right); + if (!result.isEmpty()) { + return result; + } + } + } + } + + public boolean hasNext() { + return next != null; + } + + /** + * Returns the next item and advances one {@code LiteralByteString}. + * + * @return next non-empty LiteralByteString or {@code null} + */ + public LiteralByteString next() { + if (next == null) { + throw new NoSuchElementException(); + } + LiteralByteString result = next; + next = getNextNonEmptyLeaf(); + return result; + } + + public void remove() { + throw new UnsupportedOperationException(); + } + } + + // ================================================================= + // ByteIterator + + @Override + public ByteIterator iterator() { + return new RopeByteIterator(); + } + + private class RopeByteIterator implements ByteString.ByteIterator { + + private final PieceIterator pieces; + private ByteIterator bytes; + int bytesRemaining; + + private RopeByteIterator() { + pieces = new PieceIterator(RopeByteString.this); + bytes = pieces.next().iterator(); + bytesRemaining = size(); + } + + public boolean hasNext() { + return (bytesRemaining > 0); + } + + public Byte next() { + return nextByte(); // Does not instantiate a Byte + } + + public byte nextByte() { + if (!bytes.hasNext()) { + bytes = pieces.next().iterator(); + } + --bytesRemaining; + return bytes.nextByte(); + } + + public void remove() { + throw new UnsupportedOperationException(); + } + } + + /** + * This class is the {@link RopeByteString} equivalent for + * {@link ByteArrayInputStream}. + */ + private class RopeInputStream extends InputStream { + // Iterates through the pieces of the rope + private PieceIterator pieceIterator; + // The current piece + private LiteralByteString currentPiece; + // The size of the current piece + private int currentPieceSize; + // The index of the next byte to read in the current piece + private int currentPieceIndex; + // The offset of the start of the current piece in the rope byte string + private int currentPieceOffsetInRope; + // Offset in the buffer at which user called mark(); + private int mark; + + public RopeInputStream() { + initialize(); + } + + @Override + public int read(byte b[], int offset, int length) { + if (b == null) { + throw new NullPointerException(); + } else if (offset < 0 || length < 0 || length > b.length - offset) { + throw new IndexOutOfBoundsException(); + } + return readSkipInternal(b, offset, length); + } + + @Override + public long skip(long length) { + if (length < 0) { + throw new IndexOutOfBoundsException(); + } else if (length > Integer.MAX_VALUE) { + length = Integer.MAX_VALUE; + } + return readSkipInternal(null, 0, (int) length); + } + + /** + * Internal implementation of read and skip. If b != null, then read the + * next {@code length} bytes into the buffer {@code b} at + * offset {@code offset}. If b == null, then skip the next {@code length) + * bytes. + * <p> + * This method assumes that all error checking has already happened. + * <p> + * Returns the actual number of bytes read or skipped. + */ + private int readSkipInternal(byte b[], int offset, int length) { + int bytesRemaining = length; + while (bytesRemaining > 0) { + advanceIfCurrentPieceFullyRead(); + if (currentPiece == null) { + if (bytesRemaining == length) { + // We didn't manage to read anything + return -1; + } + break; + } else { + // Copy the bytes from this piece. + int currentPieceRemaining = currentPieceSize - currentPieceIndex; + int count = Math.min(currentPieceRemaining, bytesRemaining); + if (b != null) { + currentPiece.copyTo(b, currentPieceIndex, offset, count); + offset += count; + } + currentPieceIndex += count; + bytesRemaining -= count; + } + } + // Return the number of bytes read. + return length - bytesRemaining; + } + + @Override + public int read() throws IOException { + advanceIfCurrentPieceFullyRead(); + if (currentPiece == null) { + return -1; + } else { + return currentPiece.byteAt(currentPieceIndex++) & 0xFF; + } + } + + @Override + public int available() throws IOException { + int bytesRead = currentPieceOffsetInRope + currentPieceIndex; + return RopeByteString.this.size() - bytesRead; + } + + @Override + public boolean markSupported() { + return true; + } + + @Override + public void mark(int readAheadLimit) { + // Set the mark to our position in the byte string + mark = currentPieceOffsetInRope + currentPieceIndex; + } + + @Override + public synchronized void reset() { + // Just reinitialize and skip the specified number of bytes. + initialize(); + readSkipInternal(null, 0, mark); + } + + /** Common initialization code used by both the constructor and reset() */ + private void initialize() { + pieceIterator = new PieceIterator(RopeByteString.this); + currentPiece = pieceIterator.next(); + currentPieceSize = currentPiece.size(); + currentPieceIndex = 0; + currentPieceOffsetInRope = 0; + } + + /** + * Skips to the next piece if we have read all the data in the current + * piece. Sets currentPiece to null if we have reached the end of the + * input. + */ + private void advanceIfCurrentPieceFullyRead() { + if (currentPiece != null && currentPieceIndex == currentPieceSize) { + // Generally, we can only go through this loop at most once, since + // empty strings can't end up in a rope. But better to test. + currentPieceOffsetInRope += currentPieceSize; + currentPieceIndex = 0; + if (pieceIterator.hasNext()) { + currentPiece = pieceIterator.next(); + currentPieceSize = currentPiece.size(); + } else { + currentPiece = null; + currentPieceSize = 0; + } + } + } + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/SingleFieldBuilder.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/SingleFieldBuilder.java index d4475f6..4bfc9f3 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/SingleFieldBuilder.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/SingleFieldBuilder.java @@ -31,21 +31,21 @@ package com.google.protobuf; /** - * <code>SingleFieldBuilder</code> implements a structure that a protocol + * {@code SingleFieldBuilder} implements a structure that a protocol * message uses to hold a single field of another protocol message. It supports * the classical use case of setting an immutable {@link Message} as the value * of the field and is highly optimized around this. * <br> * It also supports the additional use case of setting a {@link Message.Builder} - * as the field and deferring conversion of that <code>Builder</code> - * to an immutable <code>Message</code>. In this way, it's possible to maintain - * a tree of <code>Builder</code>'s that acts as a fully read/write data + * as the field and deferring conversion of that {@code Builder} + * to an immutable {@code Message}. In this way, it's possible to maintain + * a tree of {@code Builder}'s that acts as a fully read/write data * structure. * <br> * Logically, one can think of a tree of builders as converting the entire tree * to messages when build is called on the root or when any method is called * that desires a Message instead of a Builder. In terms of the implementation, - * the <code>SingleFieldBuilder</code> and <code>RepeatedFieldBuilder</code> + * the {@code SingleFieldBuilder} and {@code RepeatedFieldBuilder} * classes cache messages that were created so that messages only need to be * created when some change occured in its builder or a builder for one of its * descendants. @@ -99,7 +99,7 @@ public class SingleFieldBuilder /** * Get the message for the field. If the message is currently stored - * as a <code>Builder</code>, it is converted to a <code>Message</code> by + * as a {@code Builder}, it is converted to a {@code Message} by * calling {@link Message.Builder#buildPartial} on it. If no message has * been set, returns the default instance of the message. * diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/SmallSortedMap.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/SmallSortedMap.java index 1cf270f..c6cad6a 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/SmallSortedMap.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/SmallSortedMap.java @@ -51,14 +51,14 @@ import java.util.SortedMap; * remaining entries are stored in an overflow map. Iteration over the entries * in the map should be done as follows: * - * <pre> - * for (int i = 0; i < fieldMap.getNumArrayEntries(); i++) { + * <pre> {@code + * for (int i = 0; i < fieldMap.getNumArrayEntries(); i++) { * process(fieldMap.getArrayEntryAt(i)); * } - * for (Map.Entry<K, V> entry : fieldMap.getOverflowEntries()) { + * for (Map.Entry<K, V> entry : fieldMap.getOverflowEntries()) { * process(entry); * } - * </pre> + * }</pre> * * The resulting iteration is in order of ascending field tag number. The * object returned by {@link #entrySet()} adheres to the same contract but is @@ -394,7 +394,7 @@ class SmallSortedMap<K extends Comparable<K>, V> extends AbstractMap<K, V> { /** * Entry implementation that implements Comparable in order to support - * binary search witin the entry array. Also checks mutability in + * binary search within the entry array. Also checks mutability in * {@link #setValue()}. */ private class Entry implements Map.Entry<K, V>, Comparable<Entry> { diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/TextFormat.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/TextFormat.java index d5fbdab..ed46289 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/TextFormat.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/TextFormat.java @@ -55,15 +55,18 @@ import java.util.regex.Pattern; public final class TextFormat { private TextFormat() {} - private static final Printer DEFAULT_PRINTER = new Printer(false); - private static final Printer SINGLE_LINE_PRINTER = new Printer(true); + private static final Printer DEFAULT_PRINTER = new Printer(); + private static final Printer SINGLE_LINE_PRINTER = + (new Printer()).setSingleLineMode(true); + private static final Printer UNICODE_PRINTER = + (new Printer()).setEscapeNonAscii(false); /** * Outputs a textual representation of the Protocol Message supplied into * the parameter output. (This representation is the new version of the * classic "ProtocolPrinter" output from the original Protocol Buffer system) */ - public static void print(final Message message, final Appendable output) + public static void print(final MessageOrBuilder message, final Appendable output) throws IOException { DEFAULT_PRINTER.print(message, new TextGenerator(output)); } @@ -79,7 +82,7 @@ public final class TextFormat { * Generates a human readable form of this message, useful for debugging and * other purposes, with no newline characters. */ - public static String shortDebugString(final Message message) { + public static String shortDebugString(final MessageOrBuilder message) { try { final StringBuilder sb = new StringBuilder(); SINGLE_LINE_PRINTER.print(message, new TextGenerator(sb)); @@ -109,7 +112,7 @@ public final class TextFormat { * Like {@code print()}, but writes directly to a {@code String} and * returns it. */ - public static String printToString(final Message message) { + public static String printToString(final MessageOrBuilder message) { try { final StringBuilder text = new StringBuilder(); print(message, text); @@ -133,6 +136,34 @@ public final class TextFormat { } } + /** + * Same as {@code printToString()}, except that non-ASCII characters + * in string type fields are not escaped in backslash+octals. + */ + public static String printToUnicodeString(final MessageOrBuilder message) { + try { + final StringBuilder text = new StringBuilder(); + UNICODE_PRINTER.print(message, new TextGenerator(text)); + return text.toString(); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } + + /** + * Same as {@code printToString()}, except that non-ASCII characters + * in string type fields are not escaped in backslash+octals. + */ + public static String printToUnicodeString(final UnknownFieldSet fields) { + try { + final StringBuilder text = new StringBuilder(); + UNICODE_PRINTER.printUnknownFields(fields, new TextGenerator(text)); + return text.toString(); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } + public static void printField(final FieldDescriptor field, final Object value, final Appendable output) @@ -216,13 +247,26 @@ public final class TextFormat { /** Helper class for converting protobufs to text. */ private static final class Printer { /** Whether to omit newlines from the output. */ - final boolean singleLineMode; + boolean singleLineMode = false; + + /** Whether to escape non ASCII characters with backslash and octal. */ + boolean escapeNonAscii = true; + + private Printer() {} - private Printer(final boolean singleLineMode) { + /** Setter of singleLineMode */ + private Printer setSingleLineMode(boolean singleLineMode) { this.singleLineMode = singleLineMode; + return this; + } + + /** Setter of escapeNonAscii */ + private Printer setEscapeNonAscii(boolean escapeNonAscii) { + this.escapeNonAscii = escapeNonAscii; + return this; } - private void print(final Message message, final TextGenerator generator) + private void print(final MessageOrBuilder message, final TextGenerator generator) throws IOException { for (Map.Entry<FieldDescriptor, Object> field : message.getAllFields().entrySet()) { @@ -339,7 +383,9 @@ public final class TextFormat { case STRING: generator.print("\""); - generator.print(escapeText((String) value)); + generator.print(escapeNonAscii ? + escapeText((String) value) : + (String) value); generator.print("\""); break; @@ -541,7 +587,7 @@ public final class TextFormat { private int previousLine = 0; private int previousColumn = 0; - // We use possesive quantifiers (*+ and ++) because otherwise the Java + // We use possessive quantifiers (*+ and ++) because otherwise the Java // regex matcher has stack overflows on large inputs. private static final Pattern WHITESPACE = Pattern.compile("(\\s|(#.*$))++", Pattern.MULTILINE); @@ -864,7 +910,7 @@ public final class TextFormat { public ParseException parseException(final String description) { // Note: People generally prefer one-based line and column numbers. return new ParseException( - (line + 1) + ":" + (column + 1) + ": " + description); + line + 1, column + 1, description); } /** @@ -875,7 +921,7 @@ public final class TextFormat { final String description) { // Note: People generally prefer one-based line and column numbers. return new ParseException( - (previousLine + 1) + ":" + (previousColumn + 1) + ": " + description); + previousLine + 1, previousColumn + 1, description); } /** @@ -900,8 +946,45 @@ public final class TextFormat { public static class ParseException extends IOException { private static final long serialVersionUID = 3196188060225107702L; + private final int line; + private final int column; + + /** Create a new instance, with -1 as the line and column numbers. */ public ParseException(final String message) { - super(message); + this(-1, -1, message); + } + + /** + * Create a new instance + * + * @param line the line number where the parse error occurred, + * using 1-offset. + * @param column the column number where the parser error occurred, + * using 1-offset. + */ + public ParseException(final int line, final int column, + final String message) { + super(Integer.toString(line) + ":" + column + ": " + message); + this.line = line; + this.column = column; + } + + /** + * Return the line where the parse exception occurred, or -1 when + * none is provided. The value is specified as 1-offset, so the first + * line is line 1. + */ + public int getLine() { + return line; + } + + /** + * Return the column where the parse exception occurred, or -1 when + * none is provided. The value is specified as 1-offset, so the first + * line is line 1. + */ + public int getColumn() { + return column; } } @@ -1073,7 +1156,7 @@ public final class TextFormat { mergeField(tokenizer, extensionRegistry, subBuilder); } - value = subBuilder.build(); + value = subBuilder.buildPartial(); } else { tokenizer.consume(":"); @@ -1212,7 +1295,7 @@ public final class TextFormat { */ static ByteString unescapeBytes(final CharSequence charString) throws InvalidEscapeSequenceException { - // First convert the Java characater sequence to UTF-8 bytes. + // First convert the Java character sequence to UTF-8 bytes. ByteString input = ByteString.copyFromUtf8(charString.toString()); // Then unescape certain byte sequences introduced by ASCII '\\'. The valid // escapes can all be expressed with ASCII characters, so it is safe to @@ -1349,7 +1432,7 @@ public final class TextFormat { /** * Parse a 32-bit signed integer from the text. Unlike the Java standard * {@code Integer.parseInt()}, this function recognizes the prefixes "0x" - * and "0" to signify hexidecimal and octal numbers, respectively. + * and "0" to signify hexadecimal and octal numbers, respectively. */ static int parseInt32(final String text) throws NumberFormatException { return (int) parseInteger(text, true, false); @@ -1358,7 +1441,7 @@ public final class TextFormat { /** * Parse a 32-bit unsigned integer from the text. Unlike the Java standard * {@code Integer.parseInt()}, this function recognizes the prefixes "0x" - * and "0" to signify hexidecimal and octal numbers, respectively. The + * and "0" to signify hexadecimal and octal numbers, respectively. The * result is coerced to a (signed) {@code int} when returned since Java has * no unsigned integer type. */ @@ -1369,7 +1452,7 @@ public final class TextFormat { /** * Parse a 64-bit signed integer from the text. Unlike the Java standard * {@code Integer.parseInt()}, this function recognizes the prefixes "0x" - * and "0" to signify hexidecimal and octal numbers, respectively. + * and "0" to signify hexadecimal and octal numbers, respectively. */ static long parseInt64(final String text) throws NumberFormatException { return parseInteger(text, true, true); @@ -1378,7 +1461,7 @@ public final class TextFormat { /** * Parse a 64-bit unsigned integer from the text. Unlike the Java standard * {@code Integer.parseInt()}, this function recognizes the prefixes "0x" - * and "0" to signify hexidecimal and octal numbers, respectively. The + * and "0" to signify hexadecimal and octal numbers, respectively. The * result is coerced to a (signed) {@code long} when returned since Java has * no unsigned long type. */ diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/UnknownFieldSet.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/UnknownFieldSet.java index 26a15d0..45e2e6e 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/UnknownFieldSet.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/UnknownFieldSet.java @@ -46,7 +46,7 @@ import java.util.TreeMap; * {@code UnknownFieldSet} is used to keep track of fields which were seen when * parsing a protocol message but whose field numbers or types are unrecognized. * This most frequently occurs when new fields are added to a message type - * and then messages containing those feilds are read by old software that was + * and then messages containing those fields are read by old software that was * compiled before the new types were added. * * <p>Every {@link Message} contains an {@code UnknownFieldSet} (and every @@ -468,7 +468,7 @@ public final class UnknownFieldSet implements MessageLite { /** * Parse a single field from {@code input} and merge it into this set. * @param tag The field's tag number, which was already parsed. - * @return {@code false} if the tag is an engroup tag. + * @return {@code false} if the tag is an end group tag. */ public boolean mergeFieldFrom(final int tag, final CodedInputStream input) throws IOException { @@ -950,4 +950,29 @@ public final class UnknownFieldSet implements MessageLite { } } } + + /** + * Parser to implement MessageLite interface. + */ + public static final class Parser extends AbstractParser<UnknownFieldSet> { + public UnknownFieldSet parsePartialFrom( + CodedInputStream input, ExtensionRegistryLite extensionRegistry) + throws InvalidProtocolBufferException { + Builder builder = newBuilder(); + try { + builder.mergeFrom(input); + } catch (InvalidProtocolBufferException e) { + throw e.setUnfinishedMessage(builder.buildPartial()); + } catch (IOException e) { + throw new InvalidProtocolBufferException(e.getMessage()) + .setUnfinishedMessage(builder.buildPartial()); + } + return builder.buildPartial(); + } + } + + private static final Parser PARSER = new Parser(); + public final Parser getParserForType() { + return PARSER; + } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/UnmodifiableLazyStringList.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/UnmodifiableLazyStringList.java index 83e5c79..f80f0968 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/UnmodifiableLazyStringList.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/UnmodifiableLazyStringList.java @@ -32,6 +32,7 @@ package com.google.protobuf; import java.util.AbstractList; import java.util.RandomAccess; +import java.util.List; import java.util.ListIterator; import java.util.Iterator; @@ -143,4 +144,10 @@ public class UnmodifiableLazyStringList extends AbstractList<String> } }; } + + @Override + public List<?> getUnderlyingElements() { + // The returned value is already unmodifiable. + return list.getUnderlyingElements(); + } } diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/Utf8.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/Utf8.java new file mode 100644 index 0000000..388f7fc --- /dev/null +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/Utf8.java @@ -0,0 +1,349 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +/** + * A set of low-level, high-performance static utility methods related + * to the UTF-8 character encoding. This class has no dependencies + * outside of the core JDK libraries. + * + * <p>There are several variants of UTF-8. The one implemented by + * this class is the restricted definition of UTF-8 introduced in + * Unicode 3.1, which mandates the rejection of "overlong" byte + * sequences as well as rejection of 3-byte surrogate codepoint byte + * sequences. Note that the UTF-8 decoder included in Oracle's JDK + * has been modified to also reject "overlong" byte sequences, but (as + * of 2011) still accepts 3-byte surrogate codepoint byte sequences. + * + * <p>The byte sequences considered valid by this class are exactly + * those that can be roundtrip converted to Strings and back to bytes + * using the UTF-8 charset, without loss: <pre> {@code + * Arrays.equals(bytes, new String(bytes, "UTF-8").getBytes("UTF-8")) + * }</pre> + * + * <p>See the Unicode Standard,</br> + * Table 3-6. <em>UTF-8 Bit Distribution</em>,</br> + * Table 3-7. <em>Well Formed UTF-8 Byte Sequences</em>. + * + * <p>This class supports decoding of partial byte sequences, so that the + * bytes in a complete UTF-8 byte sequences can be stored in multiple + * segments. Methods typically return {@link #MALFORMED} if the partial + * byte sequence is definitely not well-formed, {@link #COMPLETE} if it is + * well-formed in the absence of additional input, or if the byte sequence + * apparently terminated in the middle of a character, an opaque integer + * "state" value containing enough information to decode the character when + * passed to a subsequent invocation of a partial decoding method. + * + * @author martinrb@google.com (Martin Buchholz) + */ +final class Utf8 { + private Utf8() {} + + /** + * State value indicating that the byte sequence is well-formed and + * complete (no further bytes are needed to complete a character). + */ + public static final int COMPLETE = 0; + + /** + * State value indicating that the byte sequence is definitely not + * well-formed. + */ + public static final int MALFORMED = -1; + + // Other state values include the partial bytes of the incomplete + // character to be decoded in the simplest way: we pack the bytes + // into the state int in little-endian order. For example: + // + // int state = byte1 ^ (byte2 << 8) ^ (byte3 << 16); + // + // Such a state is unpacked thus (note the ~ operation for byte2 to + // undo byte1's sign-extension bits): + // + // int byte1 = (byte) state; + // int byte2 = (byte) ~(state >> 8); + // int byte3 = (byte) (state >> 16); + // + // We cannot store a zero byte in the state because it would be + // indistinguishable from the absence of a byte. But we don't need + // to, because partial bytes must always be negative. When building + // a state, we ensure that byte1 is negative and subsequent bytes + // are valid trailing bytes. + + /** + * Returns {@code true} if the given byte array is a well-formed + * UTF-8 byte sequence. + * + * <p>This is a convenience method, equivalent to a call to {@code + * isValidUtf8(bytes, 0, bytes.length)}. + */ + public static boolean isValidUtf8(byte[] bytes) { + return isValidUtf8(bytes, 0, bytes.length); + } + + /** + * Returns {@code true} if the given byte array slice is a + * well-formed UTF-8 byte sequence. The range of bytes to be + * checked extends from index {@code index}, inclusive, to {@code + * limit}, exclusive. + * + * <p>This is a convenience method, equivalent to {@code + * partialIsValidUtf8(bytes, index, limit) == Utf8.COMPLETE}. + */ + public static boolean isValidUtf8(byte[] bytes, int index, int limit) { + return partialIsValidUtf8(bytes, index, limit) == COMPLETE; + } + + /** + * Tells whether the given byte array slice is a well-formed, + * malformed, or incomplete UTF-8 byte sequence. The range of bytes + * to be checked extends from index {@code index}, inclusive, to + * {@code limit}, exclusive. + * + * @param state either {@link Utf8#COMPLETE} (if this is the initial decoding + * operation) or the value returned from a call to a partial decoding method + * for the previous bytes + * + * @return {@link #MALFORMED} if the partial byte sequence is + * definitely not well-formed, {@link #COMPLETE} if it is well-formed + * (no additional input needed), or if the byte sequence is + * "incomplete", i.e. apparently terminated in the middle of a character, + * an opaque integer "state" value containing enough information to + * decode the character when passed to a subsequent invocation of a + * partial decoding method. + */ + public static int partialIsValidUtf8( + int state, byte[] bytes, int index, int limit) { + if (state != COMPLETE) { + // The previous decoding operation was incomplete (or malformed). + // We look for a well-formed sequence consisting of bytes from + // the previous decoding operation (stored in state) together + // with bytes from the array slice. + // + // We expect such "straddler characters" to be rare. + + if (index >= limit) { // No bytes? No progress. + return state; + } + int byte1 = (byte) state; + // byte1 is never ASCII. + if (byte1 < (byte) 0xE0) { + // two-byte form + + // Simultaneously checks for illegal trailing-byte in + // leading position and overlong 2-byte form. + if (byte1 < (byte) 0xC2 || + // byte2 trailing-byte test + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } else if (byte1 < (byte) 0xF0) { + // three-byte form + + // Get byte2 from saved state or array + int byte2 = (byte) ~(state >> 8); + if (byte2 == 0) { + byte2 = bytes[index++]; + if (index >= limit) { + return incompleteStateFor(byte1, byte2); + } + } + if (byte2 > (byte) 0xBF || + // overlong? 5 most significant bits must not all be zero + (byte1 == (byte) 0xE0 && byte2 < (byte) 0xA0) || + // illegal surrogate codepoint? + (byte1 == (byte) 0xED && byte2 >= (byte) 0xA0) || + // byte3 trailing-byte test + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } else { + // four-byte form + + // Get byte2 and byte3 from saved state or array + int byte2 = (byte) ~(state >> 8); + int byte3 = 0; + if (byte2 == 0) { + byte2 = bytes[index++]; + if (index >= limit) { + return incompleteStateFor(byte1, byte2); + } + } else { + byte3 = (byte) (state >> 16); + } + if (byte3 == 0) { + byte3 = bytes[index++]; + if (index >= limit) { + return incompleteStateFor(byte1, byte2, byte3); + } + } + + // If we were called with state == MALFORMED, then byte1 is 0xFF, + // which never occurs in well-formed UTF-8, and so we will return + // MALFORMED again below. + + if (byte2 > (byte) 0xBF || + // Check that 1 <= plane <= 16. Tricky optimized form of: + // if (byte1 > (byte) 0xF4 || + // byte1 == (byte) 0xF0 && byte2 < (byte) 0x90 || + // byte1 == (byte) 0xF4 && byte2 > (byte) 0x8F) + (((byte1 << 28) + (byte2 - (byte) 0x90)) >> 30) != 0 || + // byte3 trailing-byte test + byte3 > (byte) 0xBF || + // byte4 trailing-byte test + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } + } + + return partialIsValidUtf8(bytes, index, limit); + } + + /** + * Tells whether the given byte array slice is a well-formed, + * malformed, or incomplete UTF-8 byte sequence. The range of bytes + * to be checked extends from index {@code index}, inclusive, to + * {@code limit}, exclusive. + * + * <p>This is a convenience method, equivalent to a call to {@code + * partialIsValidUtf8(Utf8.COMPLETE, bytes, index, limit)}. + * + * @return {@link #MALFORMED} if the partial byte sequence is + * definitely not well-formed, {@link #COMPLETE} if it is well-formed + * (no additional input needed), or if the byte sequence is + * "incomplete", i.e. apparently terminated in the middle of a character, + * an opaque integer "state" value containing enough information to + * decode the character when passed to a subsequent invocation of a + * partial decoding method. + */ + public static int partialIsValidUtf8( + byte[] bytes, int index, int limit) { + // Optimize for 100% ASCII. + // Hotspot loves small simple top-level loops like this. + while (index < limit && bytes[index] >= 0) { + index++; + } + + return (index >= limit) ? COMPLETE : + partialIsValidUtf8NonAscii(bytes, index, limit); + } + + private static int partialIsValidUtf8NonAscii( + byte[] bytes, int index, int limit) { + for (;;) { + int byte1, byte2; + + // Optimize for interior runs of ASCII bytes. + do { + if (index >= limit) { + return COMPLETE; + } + } while ((byte1 = bytes[index++]) >= 0); + + if (byte1 < (byte) 0xE0) { + // two-byte form + + if (index >= limit) { + return byte1; + } + + // Simultaneously checks for illegal trailing-byte in + // leading position and overlong 2-byte form. + if (byte1 < (byte) 0xC2 || + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } else if (byte1 < (byte) 0xF0) { + // three-byte form + + if (index >= limit - 1) { // incomplete sequence + return incompleteStateFor(bytes, index, limit); + } + if ((byte2 = bytes[index++]) > (byte) 0xBF || + // overlong? 5 most significant bits must not all be zero + (byte1 == (byte) 0xE0 && byte2 < (byte) 0xA0) || + // check for illegal surrogate codepoints + (byte1 == (byte) 0xED && byte2 >= (byte) 0xA0) || + // byte3 trailing-byte test + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } else { + // four-byte form + + if (index >= limit - 2) { // incomplete sequence + return incompleteStateFor(bytes, index, limit); + } + if ((byte2 = bytes[index++]) > (byte) 0xBF || + // Check that 1 <= plane <= 16. Tricky optimized form of: + // if (byte1 > (byte) 0xF4 || + // byte1 == (byte) 0xF0 && byte2 < (byte) 0x90 || + // byte1 == (byte) 0xF4 && byte2 > (byte) 0x8F) + (((byte1 << 28) + (byte2 - (byte) 0x90)) >> 30) != 0 || + // byte3 trailing-byte test + bytes[index++] > (byte) 0xBF || + // byte4 trailing-byte test + bytes[index++] > (byte) 0xBF) { + return MALFORMED; + } + } + } + } + + private static int incompleteStateFor(int byte1) { + return (byte1 > (byte) 0xF4) ? + MALFORMED : byte1; + } + + private static int incompleteStateFor(int byte1, int byte2) { + return (byte1 > (byte) 0xF4 || + byte2 > (byte) 0xBF) ? + MALFORMED : byte1 ^ (byte2 << 8); + } + + private static int incompleteStateFor(int byte1, int byte2, int byte3) { + return (byte1 > (byte) 0xF4 || + byte2 > (byte) 0xBF || + byte3 > (byte) 0xBF) ? + MALFORMED : byte1 ^ (byte2 << 8) ^ (byte3 << 16); + } + + private static int incompleteStateFor(byte[] bytes, int index, int limit) { + int byte1 = bytes[index - 1]; + switch (limit - index) { + case 0: return incompleteStateFor(byte1); + case 1: return incompleteStateFor(byte1, bytes[index]); + case 2: return incompleteStateFor(byte1, bytes[index], bytes[index + 1]); + default: throw new AssertionError(); + } + } +} diff --git a/third_party/protobuf/java/src/main/java/com/google/protobuf/WireFormat.java b/third_party/protobuf/java/src/main/java/com/google/protobuf/WireFormat.java index a30f2a3..dd2d631 100644 --- a/third_party/protobuf/java/src/main/java/com/google/protobuf/WireFormat.java +++ b/third_party/protobuf/java/src/main/java/com/google/protobuf/WireFormat.java @@ -146,7 +146,7 @@ public final class WireFormat { public boolean isPackable() { return true; } } - // Field numbers for feilds in MessageSet wire format. + // Field numbers for fields in MessageSet wire format. static final int MESSAGE_SET_ITEM = 1; static final int MESSAGE_SET_TYPE_ID = 2; static final int MESSAGE_SET_MESSAGE = 3; diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/AbstractMessageTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/AbstractMessageTest.java index d53ce8d76..3d05cb7 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/AbstractMessageTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/AbstractMessageTest.java @@ -30,6 +30,7 @@ package com.google.protobuf; +import com.google.protobuf.Descriptors.FieldDescriptor; import protobuf_unittest.UnittestOptimizeFor.TestOptimizedForSize; import protobuf_unittest.UnittestProto; import protobuf_unittest.UnittestProto.ForeignMessage; @@ -167,6 +168,13 @@ public class AbstractMessageTest extends TestCase { wrappedBuilder.setUnknownFields(unknownFields); return this; } + @Override + public Message.Builder getFieldBuilder(FieldDescriptor field) { + return wrappedBuilder.getFieldBuilder(field); + } + } + public Parser<? extends Message> getParserForType() { + return wrappedMessage.getParserForType(); } } @@ -220,6 +228,34 @@ public class AbstractMessageTest extends TestCase { TestUtil.assertAllFieldsSet((TestAllTypes) message.wrappedMessage); } + public void testParsingUninitialized() throws Exception { + TestRequiredForeign.Builder builder = TestRequiredForeign.newBuilder(); + builder.getOptionalMessageBuilder().setDummy2(10); + ByteString bytes = builder.buildPartial().toByteString(); + Message.Builder abstractMessageBuilder = + new AbstractMessageWrapper.Builder(TestRequiredForeign.newBuilder()); + // mergeFrom() should not throw initialization error. + abstractMessageBuilder.mergeFrom(bytes).buildPartial(); + try { + abstractMessageBuilder.mergeFrom(bytes).build(); + fail(); + } catch (UninitializedMessageException ex) { + // pass + } + + // test DynamicMessage directly. + Message.Builder dynamicMessageBuilder = DynamicMessage.newBuilder( + TestRequiredForeign.getDescriptor()); + // mergeFrom() should not throw initialization error. + dynamicMessageBuilder.mergeFrom(bytes).buildPartial(); + try { + dynamicMessageBuilder.mergeFrom(bytes).build(); + fail(); + } catch (UninitializedMessageException ex) { + // pass + } + } + public void testPackedSerialization() throws Exception { Message abstractMessage = new AbstractMessageWrapper(TestUtil.getPackedSet()); @@ -298,12 +334,16 @@ public class AbstractMessageTest extends TestCase { new AbstractMessageWrapper.Builder(builder); assertFalse(abstractBuilder.isInitialized()); + assertEquals("a, b, c", abstractBuilder.getInitializationErrorString()); builder.setA(1); assertFalse(abstractBuilder.isInitialized()); + assertEquals("b, c", abstractBuilder.getInitializationErrorString()); builder.setB(1); assertFalse(abstractBuilder.isInitialized()); + assertEquals("c", abstractBuilder.getInitializationErrorString()); builder.setC(1); assertTrue(abstractBuilder.isInitialized()); + assertEquals("", abstractBuilder.getInitializationErrorString()); } public void testForeignIsInitialized() throws Exception { @@ -312,18 +352,27 @@ public class AbstractMessageTest extends TestCase { new AbstractMessageWrapper.Builder(builder); assertTrue(abstractBuilder.isInitialized()); + assertEquals("", abstractBuilder.getInitializationErrorString()); builder.setOptionalMessage(TEST_REQUIRED_UNINITIALIZED); assertFalse(abstractBuilder.isInitialized()); + assertEquals( + "optional_message.a, optional_message.b, optional_message.c", + abstractBuilder.getInitializationErrorString()); builder.setOptionalMessage(TEST_REQUIRED_INITIALIZED); assertTrue(abstractBuilder.isInitialized()); + assertEquals("", abstractBuilder.getInitializationErrorString()); builder.addRepeatedMessage(TEST_REQUIRED_UNINITIALIZED); assertFalse(abstractBuilder.isInitialized()); + assertEquals( + "repeated_message[0].a, repeated_message[0].b, repeated_message[0].c", + abstractBuilder.getInitializationErrorString()); builder.setRepeatedMessage(0, TEST_REQUIRED_INITIALIZED); assertTrue(abstractBuilder.isInitialized()); + assertEquals("", abstractBuilder.getInitializationErrorString()); } // ----------------------------------------------------------------- @@ -421,7 +470,7 @@ public class AbstractMessageTest extends TestCase { /** - * Asserts that the given proto has symetric equals and hashCode methods. + * Asserts that the given proto has symmetric equals and hashCode methods. */ private void checkEqualsIsConsistent(Message message) { // Object should be equal to itself. diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/BoundedByteStringTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/BoundedByteStringTest.java new file mode 100644 index 0000000..20fa2df --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/BoundedByteStringTest.java @@ -0,0 +1,68 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.UnsupportedEncodingException; + +/** + * This class tests {@link BoundedByteString}, which extends {@link LiteralByteString}, + * by inheriting the tests from {@link LiteralByteStringTest}. The only method which + * is strange enough that it needs to be overridden here is {@link #testToString()}. + * + * @author carlanton@google.com (Carl Haverl) + */ +public class BoundedByteStringTest extends LiteralByteStringTest { + + @Override + protected void setUp() throws Exception { + classUnderTest = "BoundedByteString"; + byte[] sourceBytes = ByteStringTest.getTestBytes(2341, 11337766L); + int from = 100; + int to = sourceBytes.length - 100; + stringUnderTest = ByteString.copyFrom(sourceBytes).substring(from, to); + referenceBytes = new byte[to - from]; + System.arraycopy(sourceBytes, from, referenceBytes, 0, to - from); + expectedHashCode = 727575887; + } + + @Override + public void testToString() throws UnsupportedEncodingException { + String testString = "I love unicode \u1234\u5678 characters"; + LiteralByteString unicode = new LiteralByteString(testString.getBytes(UTF_8)); + ByteString chopped = unicode.substring(2, unicode.size() - 6); + assertEquals(classUnderTest + ".substring() must have the expected type", + classUnderTest, getActualClassName(chopped)); + + String roundTripString = chopped.toString(UTF_8); + assertEquals(classUnderTest + " unicode bytes must match", + testString.substring(2, testString.length() - 6), roundTripString); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/ByteStringTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/ByteStringTest.java new file mode 100644 index 0000000..7a1d682 --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/ByteStringTest.java @@ -0,0 +1,692 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import com.google.protobuf.ByteString.Output; + +import junit.framework.TestCase; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Iterator; +import java.util.List; +import java.util.NoSuchElementException; +import java.util.Random; + +/** + * Test methods with implementations in {@link ByteString}, plus do some top-level "integration" + * tests. + * + * @author carlanton@google.com (Carl Haverl) + */ +public class ByteStringTest extends TestCase { + + private static final String UTF_16 = "UTF-16"; + + static byte[] getTestBytes(int size, long seed) { + Random random = new Random(seed); + byte[] result = new byte[size]; + random.nextBytes(result); + return result; + } + + private byte[] getTestBytes(int size) { + return getTestBytes(size, 445566L); + } + + private byte[] getTestBytes() { + return getTestBytes(1000); + } + + // Compare the entire left array with a subset of the right array. + private boolean isArrayRange(byte[] left, byte[] right, int rightOffset, int length) { + boolean stillEqual = (left.length == length); + for (int i = 0; (stillEqual && i < length); ++i) { + stillEqual = (left[i] == right[rightOffset + i]); + } + return stillEqual; + } + + // Returns true only if the given two arrays have identical contents. + private boolean isArray(byte[] left, byte[] right) { + return left.length == right.length && isArrayRange(left, right, 0, left.length); + } + + public void testSubstring_BeginIndex() { + byte[] bytes = getTestBytes(); + ByteString substring = ByteString.copyFrom(bytes).substring(500); + assertTrue("substring must contain the tail of the string", + isArrayRange(substring.toByteArray(), bytes, 500, bytes.length - 500)); + } + + public void testCopyFrom_BytesOffsetSize() { + byte[] bytes = getTestBytes(); + ByteString byteString = ByteString.copyFrom(bytes, 500, 200); + assertTrue("copyFrom sub-range must contain the expected bytes", + isArrayRange(byteString.toByteArray(), bytes, 500, 200)); + } + + public void testCopyFrom_Bytes() { + byte[] bytes = getTestBytes(); + ByteString byteString = ByteString.copyFrom(bytes); + assertTrue("copyFrom must contain the expected bytes", + isArray(byteString.toByteArray(), bytes)); + } + + public void testCopyFrom_ByteBufferSize() { + byte[] bytes = getTestBytes(); + ByteBuffer byteBuffer = ByteBuffer.allocate(bytes.length); + byteBuffer.put(bytes); + byteBuffer.position(500); + ByteString byteString = ByteString.copyFrom(byteBuffer, 200); + assertTrue("copyFrom byteBuffer sub-range must contain the expected bytes", + isArrayRange(byteString.toByteArray(), bytes, 500, 200)); + } + + public void testCopyFrom_ByteBuffer() { + byte[] bytes = getTestBytes(); + ByteBuffer byteBuffer = ByteBuffer.allocate(bytes.length); + byteBuffer.put(bytes); + byteBuffer.position(500); + ByteString byteString = ByteString.copyFrom(byteBuffer); + assertTrue("copyFrom byteBuffer sub-range must contain the expected bytes", + isArrayRange(byteString.toByteArray(), bytes, 500, bytes.length - 500)); + } + + public void testCopyFrom_StringEncoding() throws UnsupportedEncodingException { + String testString = "I love unicode \u1234\u5678 characters"; + ByteString byteString = ByteString.copyFrom(testString, UTF_16); + byte[] testBytes = testString.getBytes(UTF_16); + assertTrue("copyFrom string must respect the charset", + isArrayRange(byteString.toByteArray(), testBytes, 0, testBytes.length)); + } + + public void testCopyFrom_Utf8() throws UnsupportedEncodingException { + String testString = "I love unicode \u1234\u5678 characters"; + ByteString byteString = ByteString.copyFromUtf8(testString); + byte[] testBytes = testString.getBytes("UTF-8"); + assertTrue("copyFromUtf8 string must respect the charset", + isArrayRange(byteString.toByteArray(), testBytes, 0, testBytes.length)); + } + + public void testCopyFrom_Iterable() { + byte[] testBytes = getTestBytes(77777, 113344L); + final List<ByteString> pieces = makeConcretePieces(testBytes); + // Call copyFrom() on a Collection + ByteString byteString = ByteString.copyFrom(pieces); + assertTrue("copyFrom a List must contain the expected bytes", + isArrayRange(byteString.toByteArray(), testBytes, 0, testBytes.length)); + // Call copyFrom on an iteration that's not a collection + ByteString byteStringAlt = ByteString.copyFrom(new Iterable<ByteString>() { + public Iterator<ByteString> iterator() { + return pieces.iterator(); + } + }); + assertEquals("copyFrom from an Iteration must contain the expected bytes", + byteString, byteStringAlt); + } + + public void testCopyTo_TargetOffset() { + byte[] bytes = getTestBytes(); + ByteString byteString = ByteString.copyFrom(bytes); + byte[] target = new byte[bytes.length + 1000]; + byteString.copyTo(target, 400); + assertTrue("copyFrom byteBuffer sub-range must contain the expected bytes", + isArrayRange(bytes, target, 400, bytes.length)); + } + + public void testReadFrom_emptyStream() throws IOException { + ByteString byteString = + ByteString.readFrom(new ByteArrayInputStream(new byte[0])); + assertSame("reading an empty stream must result in the EMPTY constant " + + "byte string", ByteString.EMPTY, byteString); + } + + public void testReadFrom_smallStream() throws IOException { + assertReadFrom(getTestBytes(10)); + } + + public void testReadFrom_mutating() throws IOException { + byte[] capturedArray = null; + EvilInputStream eis = new EvilInputStream(); + ByteString byteString = ByteString.readFrom(eis); + + capturedArray = eis.capturedArray; + byte[] originalValue = byteString.toByteArray(); + for (int x = 0; x < capturedArray.length; ++x) { + capturedArray[x] = (byte) 0; + } + + byte[] newValue = byteString.toByteArray(); + assertTrue("copyFrom byteBuffer must not grant access to underlying array", + Arrays.equals(originalValue, newValue)); + } + + // Tests sizes that are near the rope copy-out threshold. + public void testReadFrom_mediumStream() throws IOException { + assertReadFrom(getTestBytes(ByteString.CONCATENATE_BY_COPY_SIZE - 1)); + assertReadFrom(getTestBytes(ByteString.CONCATENATE_BY_COPY_SIZE)); + assertReadFrom(getTestBytes(ByteString.CONCATENATE_BY_COPY_SIZE + 1)); + assertReadFrom(getTestBytes(200)); + } + + // Tests sizes that are over multi-segment rope threshold. + public void testReadFrom_largeStream() throws IOException { + assertReadFrom(getTestBytes(0x100)); + assertReadFrom(getTestBytes(0x101)); + assertReadFrom(getTestBytes(0x110)); + assertReadFrom(getTestBytes(0x1000)); + assertReadFrom(getTestBytes(0x1001)); + assertReadFrom(getTestBytes(0x1010)); + assertReadFrom(getTestBytes(0x10000)); + assertReadFrom(getTestBytes(0x10001)); + assertReadFrom(getTestBytes(0x10010)); + } + + // Tests sizes that are near the read buffer size. + public void testReadFrom_byteBoundaries() throws IOException { + final int min = ByteString.MIN_READ_FROM_CHUNK_SIZE; + final int max = ByteString.MAX_READ_FROM_CHUNK_SIZE; + + assertReadFrom(getTestBytes(min - 1)); + assertReadFrom(getTestBytes(min)); + assertReadFrom(getTestBytes(min + 1)); + + assertReadFrom(getTestBytes(min * 2 - 1)); + assertReadFrom(getTestBytes(min * 2)); + assertReadFrom(getTestBytes(min * 2 + 1)); + + assertReadFrom(getTestBytes(min * 4 - 1)); + assertReadFrom(getTestBytes(min * 4)); + assertReadFrom(getTestBytes(min * 4 + 1)); + + assertReadFrom(getTestBytes(min * 8 - 1)); + assertReadFrom(getTestBytes(min * 8)); + assertReadFrom(getTestBytes(min * 8 + 1)); + + assertReadFrom(getTestBytes(max - 1)); + assertReadFrom(getTestBytes(max)); + assertReadFrom(getTestBytes(max + 1)); + + assertReadFrom(getTestBytes(max * 2 - 1)); + assertReadFrom(getTestBytes(max * 2)); + assertReadFrom(getTestBytes(max * 2 + 1)); + } + + // Tests that IOExceptions propagate through ByteString.readFrom(). + public void testReadFrom_IOExceptions() { + try { + ByteString.readFrom(new FailStream()); + fail("readFrom must throw the underlying IOException"); + + } catch (IOException e) { + assertEquals("readFrom must throw the expected exception", + "synthetic failure", e.getMessage()); + } + } + + // Tests that ByteString.readFrom works with streams that don't + // always fill their buffers. + public void testReadFrom_reluctantStream() throws IOException { + final byte[] data = getTestBytes(0x1000); + + ByteString byteString = ByteString.readFrom(new ReluctantStream(data)); + assertTrue("readFrom byte stream must contain the expected bytes", + isArray(byteString.toByteArray(), data)); + + // Same test as above, but with some specific chunk sizes. + assertReadFromReluctantStream(data, 100); + assertReadFromReluctantStream(data, 248); + assertReadFromReluctantStream(data, 249); + assertReadFromReluctantStream(data, 250); + assertReadFromReluctantStream(data, 251); + assertReadFromReluctantStream(data, 0x1000); + assertReadFromReluctantStream(data, 0x1001); + } + + // Fails unless ByteString.readFrom reads the bytes correctly from a + // reluctant stream with the given chunkSize parameter. + private void assertReadFromReluctantStream(byte[] bytes, int chunkSize) + throws IOException { + ByteString b = ByteString.readFrom(new ReluctantStream(bytes), chunkSize); + assertTrue("readFrom byte stream must contain the expected bytes", + isArray(b.toByteArray(), bytes)); + } + + // Tests that ByteString.readFrom works with streams that implement + // available(). + public void testReadFrom_available() throws IOException { + final byte[] data = getTestBytes(0x1001); + + ByteString byteString = ByteString.readFrom(new AvailableStream(data)); + assertTrue("readFrom byte stream must contain the expected bytes", + isArray(byteString.toByteArray(), data)); + } + + // Fails unless ByteString.readFrom reads the bytes correctly. + private void assertReadFrom(byte[] bytes) throws IOException { + ByteString byteString = + ByteString.readFrom(new ByteArrayInputStream(bytes)); + assertTrue("readFrom byte stream must contain the expected bytes", + isArray(byteString.toByteArray(), bytes)); + } + + // A stream that fails when read. + private static final class FailStream extends InputStream { + @Override public int read() throws IOException { + throw new IOException("synthetic failure"); + } + } + + // A stream that simulates blocking by only producing 250 characters + // per call to read(byte[]). + private static class ReluctantStream extends InputStream { + protected final byte[] data; + protected int pos = 0; + + public ReluctantStream(byte[] data) { + this.data = data; + } + + @Override public int read() { + if (pos == data.length) { + return -1; + } else { + return data[pos++]; + } + } + + @Override public int read(byte[] buf) { + return read(buf, 0, buf.length); + } + + @Override public int read(byte[] buf, int offset, int size) { + if (pos == data.length) { + return -1; + } + int count = Math.min(Math.min(size, data.length - pos), 250); + System.arraycopy(data, pos, buf, offset, count); + pos += count; + return count; + } + } + + // Same as above, but also implements available(). + private static final class AvailableStream extends ReluctantStream { + public AvailableStream(byte[] data) { + super(data); + } + + @Override public int available() { + return Math.min(250, data.length - pos); + } + } + + // A stream which exposes the byte array passed into read(byte[], int, int). + private static class EvilInputStream extends InputStream { + public byte[] capturedArray = null; + + @Override + public int read(byte[] buf, int off, int len) { + if (capturedArray != null) { + return -1; + } else { + capturedArray = buf; + for (int x = 0; x < len; ++x) { + buf[x] = (byte) x; + } + return len; + } + } + + @Override + public int read() { + // Purposefully do nothing. + return -1; + } + } + + // A stream which exposes the byte array passed into write(byte[], int, int). + private static class EvilOutputStream extends OutputStream { + public byte[] capturedArray = null; + + @Override + public void write(byte[] buf, int off, int len) { + if (capturedArray == null) { + capturedArray = buf; + } + } + + @Override + public void write(int ignored) { + // Purposefully do nothing. + } + } + + public void testToStringUtf8() throws UnsupportedEncodingException { + String testString = "I love unicode \u1234\u5678 characters"; + byte[] testBytes = testString.getBytes("UTF-8"); + ByteString byteString = ByteString.copyFrom(testBytes); + assertEquals("copyToStringUtf8 must respect the charset", + testString, byteString.toStringUtf8()); + } + + public void testNewOutput_InitialCapacity() throws IOException { + byte[] bytes = getTestBytes(); + ByteString.Output output = ByteString.newOutput(bytes.length + 100); + output.write(bytes); + ByteString byteString = output.toByteString(); + assertTrue( + "String built from newOutput(int) must contain the expected bytes", + isArrayRange(bytes, byteString.toByteArray(), 0, bytes.length)); + } + + // Test newOutput() using a variety of buffer sizes and a variety of (fixed) + // write sizes + public void testNewOutput_ArrayWrite() throws IOException { + byte[] bytes = getTestBytes(); + int length = bytes.length; + int[] bufferSizes = {128, 256, length / 2, length - 1, length, length + 1, + 2 * length, 3 * length}; + int[] writeSizes = {1, 4, 5, 7, 23, bytes.length}; + + for (int bufferSize : bufferSizes) { + for (int writeSize : writeSizes) { + // Test writing the entire output writeSize bytes at a time. + ByteString.Output output = ByteString.newOutput(bufferSize); + for (int i = 0; i < length; i += writeSize) { + output.write(bytes, i, Math.min(writeSize, length - i)); + } + ByteString byteString = output.toByteString(); + assertTrue("String built from newOutput() must contain the expected bytes", + isArrayRange(bytes, byteString.toByteArray(), 0, bytes.length)); + } + } + } + + // Test newOutput() using a variety of buffer sizes, but writing all the + // characters using write(byte); + public void testNewOutput_WriteChar() throws IOException { + byte[] bytes = getTestBytes(); + int length = bytes.length; + int[] bufferSizes = {0, 1, 128, 256, length / 2, + length - 1, length, length + 1, + 2 * length, 3 * length}; + for (int bufferSize : bufferSizes) { + ByteString.Output output = ByteString.newOutput(bufferSize); + for (byte byteValue : bytes) { + output.write(byteValue); + } + ByteString byteString = output.toByteString(); + assertTrue("String built from newOutput() must contain the expected bytes", + isArrayRange(bytes, byteString.toByteArray(), 0, bytes.length)); + } + } + + // Test newOutput() in which we write the bytes using a variety of methods + // and sizes, and in which we repeatedly call toByteString() in the middle. + public void testNewOutput_Mixed() throws IOException { + Random rng = new Random(1); + byte[] bytes = getTestBytes(); + int length = bytes.length; + int[] bufferSizes = {0, 1, 128, 256, length / 2, + length - 1, length, length + 1, + 2 * length, 3 * length}; + + for (int bufferSize : bufferSizes) { + // Test writing the entire output using a mixture of write sizes and + // methods; + ByteString.Output output = ByteString.newOutput(bufferSize); + int position = 0; + while (position < bytes.length) { + if (rng.nextBoolean()) { + int count = 1 + rng.nextInt(bytes.length - position); + output.write(bytes, position, count); + position += count; + } else { + output.write(bytes[position]); + position++; + } + assertEquals("size() returns the right value", position, output.size()); + assertTrue("newOutput() substring must have correct bytes", + isArrayRange(output.toByteString().toByteArray(), + bytes, 0, position)); + } + ByteString byteString = output.toByteString(); + assertTrue("String built from newOutput() must contain the expected bytes", + isArrayRange(bytes, byteString.toByteArray(), 0, bytes.length)); + } + } + + public void testNewOutputEmpty() throws IOException { + // Make sure newOutput() correctly builds empty byte strings + ByteString byteString = ByteString.newOutput().toByteString(); + assertEquals(ByteString.EMPTY, byteString); + } + + public void testNewOutput_Mutating() throws IOException { + Output os = ByteString.newOutput(5); + os.write(new byte[] {1, 2, 3, 4, 5}); + EvilOutputStream eos = new EvilOutputStream(); + os.writeTo(eos); + byte[] capturedArray = eos.capturedArray; + ByteString byteString = os.toByteString(); + byte[] oldValue = byteString.toByteArray(); + Arrays.fill(capturedArray, (byte) 0); + byte[] newValue = byteString.toByteArray(); + assertTrue("Output must not provide access to the underlying byte array", + Arrays.equals(oldValue, newValue)); + } + + public void testNewCodedBuilder() throws IOException { + byte[] bytes = getTestBytes(); + ByteString.CodedBuilder builder = ByteString.newCodedBuilder(bytes.length); + builder.getCodedOutput().writeRawBytes(bytes); + ByteString byteString = builder.build(); + assertTrue("String built from newCodedBuilder() must contain the expected bytes", + isArrayRange(bytes, byteString.toByteArray(), 0, bytes.length)); + } + + public void testSubstringParity() { + byte[] bigBytes = getTestBytes(2048 * 1024, 113344L); + int start = 512 * 1024 - 3333; + int end = 512 * 1024 + 7777; + ByteString concreteSubstring = ByteString.copyFrom(bigBytes).substring(start, end); + boolean ok = true; + for (int i = start; ok && i < end; ++i) { + ok = (bigBytes[i] == concreteSubstring.byteAt(i - start)); + } + assertTrue("Concrete substring didn't capture the right bytes", ok); + + ByteString literalString = ByteString.copyFrom(bigBytes, start, end - start); + assertTrue("Substring must be equal to literal string", + concreteSubstring.equals(literalString)); + assertEquals("Substring must have same hashcode as literal string", + literalString.hashCode(), concreteSubstring.hashCode()); + } + + public void testCompositeSubstring() { + byte[] referenceBytes = getTestBytes(77748, 113344L); + + List<ByteString> pieces = makeConcretePieces(referenceBytes); + ByteString listString = ByteString.copyFrom(pieces); + + int from = 1000; + int to = 40000; + ByteString compositeSubstring = listString.substring(from, to); + byte[] substringBytes = compositeSubstring.toByteArray(); + boolean stillEqual = true; + for (int i = 0; stillEqual && i < to - from; ++i) { + stillEqual = referenceBytes[from + i] == substringBytes[i]; + } + assertTrue("Substring must return correct bytes", stillEqual); + + stillEqual = true; + for (int i = 0; stillEqual && i < to - from; ++i) { + stillEqual = referenceBytes[from + i] == compositeSubstring.byteAt(i); + } + assertTrue("Substring must support byteAt() correctly", stillEqual); + + ByteString literalSubstring = ByteString.copyFrom(referenceBytes, from, to - from); + assertTrue("Composite substring must equal a literal substring over the same bytes", + compositeSubstring.equals(literalSubstring)); + assertTrue("Literal substring must equal a composite substring over the same bytes", + literalSubstring.equals(compositeSubstring)); + + assertEquals("We must get the same hashcodes for composite and literal substrings", + literalSubstring.hashCode(), compositeSubstring.hashCode()); + + assertFalse("We can't be equal to a proper substring", + compositeSubstring.equals(literalSubstring.substring(0, literalSubstring.size() - 1))); + } + + public void testCopyFromList() { + byte[] referenceBytes = getTestBytes(77748, 113344L); + ByteString literalString = ByteString.copyFrom(referenceBytes); + + List<ByteString> pieces = makeConcretePieces(referenceBytes); + ByteString listString = ByteString.copyFrom(pieces); + + assertTrue("Composite string must be equal to literal string", + listString.equals(literalString)); + assertEquals("Composite string must have same hashcode as literal string", + literalString.hashCode(), listString.hashCode()); + } + + public void testConcat() { + byte[] referenceBytes = getTestBytes(77748, 113344L); + ByteString literalString = ByteString.copyFrom(referenceBytes); + + List<ByteString> pieces = makeConcretePieces(referenceBytes); + + Iterator<ByteString> iter = pieces.iterator(); + ByteString concatenatedString = iter.next(); + while (iter.hasNext()) { + concatenatedString = concatenatedString.concat(iter.next()); + } + + assertTrue("Concatenated string must be equal to literal string", + concatenatedString.equals(literalString)); + assertEquals("Concatenated string must have same hashcode as literal string", + literalString.hashCode(), concatenatedString.hashCode()); + } + + /** + * Test the Rope implementation can deal with Empty nodes, even though we + * guard against them. See also {@link LiteralByteStringTest#testConcat_empty()}. + */ + public void testConcat_empty() { + byte[] referenceBytes = getTestBytes(7748, 113344L); + ByteString literalString = ByteString.copyFrom(referenceBytes); + + ByteString duo = RopeByteString.newInstanceForTest(literalString, literalString); + ByteString temp = RopeByteString.newInstanceForTest( + RopeByteString.newInstanceForTest(literalString, ByteString.EMPTY), + RopeByteString.newInstanceForTest(ByteString.EMPTY, literalString)); + ByteString quintet = RopeByteString.newInstanceForTest(temp, ByteString.EMPTY); + + assertTrue("String with concatenated nulls must equal simple concatenate", + duo.equals(quintet)); + assertEquals("String with concatenated nulls have same hashcode as simple concatenate", + duo.hashCode(), quintet.hashCode()); + + ByteString.ByteIterator duoIter = duo.iterator(); + ByteString.ByteIterator quintetIter = quintet.iterator(); + boolean stillEqual = true; + while (stillEqual && quintetIter.hasNext()) { + stillEqual = (duoIter.nextByte() == quintetIter.nextByte()); + } + assertTrue("We must get the same characters by iterating", stillEqual); + assertFalse("Iterator must be exhausted", duoIter.hasNext()); + try { + duoIter.nextByte(); + fail("Should have thrown an exception."); + } catch (NoSuchElementException e) { + // This is success + } + try { + quintetIter.nextByte(); + fail("Should have thrown an exception."); + } catch (NoSuchElementException e) { + // This is success + } + + // Test that even if we force empty strings in as rope leaves in this + // configuration, we always get a (possibly Bounded) LiteralByteString + // for a length 1 substring. + // + // It is possible, using the testing factory method to create deeply nested + // trees of empty leaves, to make a string that will fail this test. + for (int i = 1; i < duo.size(); ++i) { + assertTrue("Substrings of size() < 2 must not be RopeByteStrings", + duo.substring(i - 1, i) instanceof LiteralByteString); + } + for (int i = 1; i < quintet.size(); ++i) { + assertTrue("Substrings of size() < 2 must not be RopeByteStrings", + quintet.substring(i - 1, i) instanceof LiteralByteString); + } + } + + public void testStartsWith() { + byte[] bytes = getTestBytes(1000, 1234L); + ByteString string = ByteString.copyFrom(bytes); + ByteString prefix = ByteString.copyFrom(bytes, 0, 500); + ByteString suffix = ByteString.copyFrom(bytes, 400, 600); + assertTrue(string.startsWith(ByteString.EMPTY)); + assertTrue(string.startsWith(string)); + assertTrue(string.startsWith(prefix)); + assertFalse(string.startsWith(suffix)); + assertFalse(prefix.startsWith(suffix)); + assertFalse(suffix.startsWith(prefix)); + assertFalse(ByteString.EMPTY.startsWith(prefix)); + assertTrue(ByteString.EMPTY.startsWith(ByteString.EMPTY)); + } + + static List<ByteString> makeConcretePieces(byte[] referenceBytes) { + List<ByteString> pieces = new ArrayList<ByteString>(); + // Starting length should be small enough that we'll do some concatenating by + // copying if we just concatenate all these pieces together. + for (int start = 0, length = 16; start < referenceBytes.length; start += length) { + length = (length << 1) - 1; + if (start + length > referenceBytes.length) { + length = referenceBytes.length - start; + } + pieces.add(ByteString.copyFrom(referenceBytes, start, length)); + } + return pieces; + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/CodedOutputStreamTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/CodedOutputStreamTest.java index 74476e3..8625502 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/CodedOutputStreamTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/CodedOutputStreamTest.java @@ -30,8 +30,10 @@ package com.google.protobuf; +import protobuf_unittest.UnittestProto.SparseEnumMessage; import protobuf_unittest.UnittestProto.TestAllTypes; import protobuf_unittest.UnittestProto.TestPackedTypes; +import protobuf_unittest.UnittestProto.TestSparseEnum; import junit.framework.TestCase; @@ -302,17 +304,14 @@ public class CodedOutputStreamTest extends TestCase { } /** Test writing a message containing a negative enum value. This used to - * fail because the size was not properly computed as a sign-extended varint. */ + * fail because the size was not properly computed as a sign-extended varint. + */ public void testWriteMessageWithNegativeEnumValue() throws Exception { - protobuf_unittest.UnittestProto.SparseEnumMessage message = - protobuf_unittest.UnittestProto.SparseEnumMessage.newBuilder() - .setSparseEnum(protobuf_unittest.UnittestProto.TestSparseEnum.SPARSE_E) - .build(); + SparseEnumMessage message = SparseEnumMessage.newBuilder() + .setSparseEnum(TestSparseEnum.SPARSE_E) .build(); assertTrue(message.getSparseEnum().getNumber() < 0); byte[] rawBytes = message.toByteArray(); - protobuf_unittest.UnittestProto.SparseEnumMessage message2 = - protobuf_unittest.UnittestProto.SparseEnumMessage.parseFrom(rawBytes); - assertEquals(protobuf_unittest.UnittestProto.TestSparseEnum.SPARSE_E, - message2.getSparseEnum()); + SparseEnumMessage message2 = SparseEnumMessage.parseFrom(rawBytes); + assertEquals(TestSparseEnum.SPARSE_E, message2.getSparseEnum()); } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/DescriptorsTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/DescriptorsTest.java index 65d06e3..9c31091 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/DescriptorsTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/DescriptorsTest.java @@ -31,6 +31,8 @@ package com.google.protobuf; import com.google.protobuf.DescriptorProtos.DescriptorProto; +import com.google.protobuf.DescriptorProtos.EnumDescriptorProto; +import com.google.protobuf.DescriptorProtos.EnumValueDescriptorProto; import com.google.protobuf.DescriptorProtos.FieldDescriptorProto; import com.google.protobuf.DescriptorProtos.FileDescriptorProto; import com.google.protobuf.Descriptors.DescriptorValidationException; @@ -60,6 +62,7 @@ import junit.framework.TestCase; import java.util.Arrays; import java.util.Collections; +import java.util.List; /** * Unit test for {@link Descriptors}. @@ -426,7 +429,7 @@ public class DescriptorsTest extends TestCase { UnittestEnormousDescriptor.getDescriptor() .toProto().getSerializedSize() > 65536); } - + /** * Tests that the DescriptorValidationException works as intended. */ @@ -445,7 +448,7 @@ public class DescriptorsTest extends TestCase { .build()) .build(); try { - Descriptors.FileDescriptor.buildFrom(fileDescriptorProto, + Descriptors.FileDescriptor.buildFrom(fileDescriptorProto, new FileDescriptor[0]); fail("DescriptorValidationException expected"); } catch (DescriptorValidationException e) { @@ -457,4 +460,189 @@ public class DescriptorsTest extends TestCase { assertTrue(e.getCause().getMessage().indexOf("invalid") != -1); } } + + /** + * Tests the translate/crosslink for an example where a message field's name + * and type name are the same. + */ + public void testDescriptorComplexCrosslink() throws Exception { + FileDescriptorProto fileDescriptorProto = FileDescriptorProto.newBuilder() + .setName("foo.proto") + .addMessageType(DescriptorProto.newBuilder() + .setName("Foo") + .addField(FieldDescriptorProto.newBuilder() + .setLabel(FieldDescriptorProto.Label.LABEL_OPTIONAL) + .setType(FieldDescriptorProto.Type.TYPE_INT32) + .setName("foo") + .setNumber(1) + .build()) + .build()) + .addMessageType(DescriptorProto.newBuilder() + .setName("Bar") + .addField(FieldDescriptorProto.newBuilder() + .setLabel(FieldDescriptorProto.Label.LABEL_OPTIONAL) + .setTypeName("Foo") + .setName("Foo") + .setNumber(1) + .build()) + .build()) + .build(); + // translate and crosslink + FileDescriptor file = + Descriptors.FileDescriptor.buildFrom(fileDescriptorProto, + new FileDescriptor[0]); + // verify resulting descriptors + assertNotNull(file); + List<Descriptor> msglist = file.getMessageTypes(); + assertNotNull(msglist); + assertTrue(msglist.size() == 2); + boolean barFound = false; + for (Descriptor desc : msglist) { + if (desc.getName().equals("Bar")) { + barFound = true; + assertNotNull(desc.getFields()); + List<FieldDescriptor> fieldlist = desc.getFields(); + assertNotNull(fieldlist); + assertTrue(fieldlist.size() == 1); + assertTrue(fieldlist.get(0).getType() == FieldDescriptor.Type.MESSAGE); + assertTrue(fieldlist.get(0).getMessageType().getName().equals("Foo")); + } + } + assertTrue(barFound); + } + + public void testInvalidPublicDependency() throws Exception { + FileDescriptorProto fooProto = FileDescriptorProto.newBuilder() + .setName("foo.proto") .build(); + FileDescriptorProto barProto = FileDescriptorProto.newBuilder() + .setName("boo.proto") + .addDependency("foo.proto") + .addPublicDependency(1) // Error, should be 0. + .build(); + FileDescriptor fooFile = Descriptors.FileDescriptor.buildFrom(fooProto, + new FileDescriptor[0]); + try { + Descriptors.FileDescriptor.buildFrom(barProto, + new FileDescriptor[] {fooFile}); + fail("DescriptorValidationException expected"); + } catch (DescriptorValidationException e) { + assertTrue( + e.getMessage().indexOf("Invalid public dependency index.") != -1); + } + } + + public void testHiddenDependency() throws Exception { + FileDescriptorProto barProto = FileDescriptorProto.newBuilder() + .setName("bar.proto") + .addMessageType(DescriptorProto.newBuilder().setName("Bar")) + .build(); + FileDescriptorProto forwardProto = FileDescriptorProto.newBuilder() + .setName("forward.proto") + .addDependency("bar.proto") + .build(); + FileDescriptorProto fooProto = FileDescriptorProto.newBuilder() + .setName("foo.proto") + .addDependency("forward.proto") + .addMessageType(DescriptorProto.newBuilder() + .setName("Foo") + .addField(FieldDescriptorProto.newBuilder() + .setLabel(FieldDescriptorProto.Label.LABEL_OPTIONAL) + .setTypeName("Bar") + .setName("bar") + .setNumber(1))) + .build(); + FileDescriptor barFile = Descriptors.FileDescriptor.buildFrom( + barProto, new FileDescriptor[0]); + FileDescriptor forwardFile = Descriptors.FileDescriptor.buildFrom( + forwardProto, new FileDescriptor[] {barFile}); + + try { + Descriptors.FileDescriptor.buildFrom( + fooProto, new FileDescriptor[] {forwardFile}); + fail("DescriptorValidationException expected"); + } catch (DescriptorValidationException e) { + assertTrue(e.getMessage().indexOf("Bar") != -1); + assertTrue(e.getMessage().indexOf("is not defined") != -1); + } + } + + public void testPublicDependency() throws Exception { + FileDescriptorProto barProto = FileDescriptorProto.newBuilder() + .setName("bar.proto") + .addMessageType(DescriptorProto.newBuilder().setName("Bar")) + .build(); + FileDescriptorProto forwardProto = FileDescriptorProto.newBuilder() + .setName("forward.proto") + .addDependency("bar.proto") + .addPublicDependency(0) + .build(); + FileDescriptorProto fooProto = FileDescriptorProto.newBuilder() + .setName("foo.proto") + .addDependency("forward.proto") + .addMessageType(DescriptorProto.newBuilder() + .setName("Foo") + .addField(FieldDescriptorProto.newBuilder() + .setLabel(FieldDescriptorProto.Label.LABEL_OPTIONAL) + .setTypeName("Bar") + .setName("bar") + .setNumber(1))) + .build(); + FileDescriptor barFile = Descriptors.FileDescriptor.buildFrom( + barProto, new FileDescriptor[0]); + FileDescriptor forwardFile = Descriptors.FileDescriptor.buildFrom( + forwardProto, new FileDescriptor[]{barFile}); + Descriptors.FileDescriptor.buildFrom( + fooProto, new FileDescriptor[] {forwardFile}); + } + + /** + * Tests the translate/crosslink for an example with a more complex namespace + * referencing. + */ + public void testComplexNamespacePublicDependency() throws Exception { + FileDescriptorProto fooProto = FileDescriptorProto.newBuilder() + .setName("bar.proto") + .setPackage("a.b.c.d.bar.shared") + .addEnumType(EnumDescriptorProto.newBuilder() + .setName("MyEnum") + .addValue(EnumValueDescriptorProto.newBuilder() + .setName("BLAH") + .setNumber(1))) + .build(); + FileDescriptorProto barProto = FileDescriptorProto.newBuilder() + .setName("foo.proto") + .addDependency("bar.proto") + .setPackage("a.b.c.d.foo.shared") + .addMessageType(DescriptorProto.newBuilder() + .setName("MyMessage") + .addField(FieldDescriptorProto.newBuilder() + .setLabel(FieldDescriptorProto.Label.LABEL_REPEATED) + .setTypeName("bar.shared.MyEnum") + .setName("MyField") + .setNumber(1))) + .build(); + // translate and crosslink + FileDescriptor fooFile = Descriptors.FileDescriptor.buildFrom( + fooProto, new FileDescriptor[0]); + FileDescriptor barFile = Descriptors.FileDescriptor.buildFrom( + barProto, new FileDescriptor[]{fooFile}); + // verify resulting descriptors + assertNotNull(barFile); + List<Descriptor> msglist = barFile.getMessageTypes(); + assertNotNull(msglist); + assertTrue(msglist.size() == 1); + Descriptor desc = msglist.get(0); + if (desc.getName().equals("MyMessage")) { + assertNotNull(desc.getFields()); + List<FieldDescriptor> fieldlist = desc.getFields(); + assertNotNull(fieldlist); + assertTrue(fieldlist.size() == 1); + FieldDescriptor field = fieldlist.get(0); + assertTrue(field.getType() == FieldDescriptor.Type.ENUM); + assertTrue(field.getEnumType().getName().equals("MyEnum")); + assertTrue(field.getEnumType().getFile().getName().equals("bar.proto")); + assertTrue(field.getEnumType().getFile().getPackage().equals( + "a.b.c.d.bar.shared")); + } + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/DynamicMessageTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/DynamicMessageTest.java index aabccda..990e8ca 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/DynamicMessageTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/DynamicMessageTest.java @@ -30,8 +30,9 @@ package com.google.protobuf; -import protobuf_unittest.UnittestProto.TestAllTypes; import protobuf_unittest.UnittestProto.TestAllExtensions; +import protobuf_unittest.UnittestProto.TestAllTypes; +import protobuf_unittest.UnittestProto.TestEmptyMessage; import protobuf_unittest.UnittestProto.TestPackedTypes; import junit.framework.TestCase; @@ -61,28 +62,44 @@ public class DynamicMessageTest extends TestCase { reflectionTester.assertAllFieldsSetViaReflection(message); } - public void testDoubleBuildError() throws Exception { + public void testSettersAfterBuild() throws Exception { Message.Builder builder = DynamicMessage.newBuilder(TestAllTypes.getDescriptor()); + Message firstMessage = builder.build(); + // double build() builder.build(); - try { - builder.build(); - fail("Should have thrown exception."); - } catch (IllegalStateException e) { - // Success. - } + // clear() after build() + builder.clear(); + // setters after build() + reflectionTester.setAllFieldsViaReflection(builder); + Message message = builder.build(); + reflectionTester.assertAllFieldsSetViaReflection(message); + // repeated setters after build() + reflectionTester.modifyRepeatedFieldsViaReflection(builder); + message = builder.build(); + reflectionTester.assertRepeatedFieldsModifiedViaReflection(message); + // firstMessage shouldn't have been modified. + reflectionTester.assertClearViaReflection(firstMessage); } - public void testClearAfterBuildError() throws Exception { + public void testUnknownFields() throws Exception { Message.Builder builder = - DynamicMessage.newBuilder(TestAllTypes.getDescriptor()); - builder.build(); - try { - builder.clear(); - fail("Should have thrown exception."); - } catch (IllegalStateException e) { - // Success. - } + DynamicMessage.newBuilder(TestEmptyMessage.getDescriptor()); + builder.setUnknownFields(UnknownFieldSet.newBuilder() + .addField(1, UnknownFieldSet.Field.newBuilder().addVarint(1).build()) + .addField(2, UnknownFieldSet.Field.newBuilder().addFixed32(1).build()) + .build()); + Message message = builder.build(); + assertEquals(2, message.getUnknownFields().asMap().size()); + // clone() with unknown fields + Message.Builder newBuilder = builder.clone(); + assertEquals(2, newBuilder.getUnknownFields().asMap().size()); + // clear() with unknown fields + newBuilder.clear(); + assertTrue(newBuilder.getUnknownFields().asMap().isEmpty()); + // serialize/parse with unknown fields + newBuilder.mergeFrom(message.toByteString()); + assertEquals(2, newBuilder.getUnknownFields().asMap().size()); } public void testDynamicMessageSettersRejectNull() throws Exception { @@ -167,6 +184,23 @@ public class DynamicMessageTest extends TestCase { Message message2 = DynamicMessage.parseFrom(TestAllTypes.getDescriptor(), rawBytes); reflectionTester.assertAllFieldsSetViaReflection(message2); + + // Test Parser interface. + Message message3 = message2.getParserForType().parseFrom(rawBytes); + reflectionTester.assertAllFieldsSetViaReflection(message3); + } + + public void testDynamicMessageExtensionParsing() throws Exception { + ByteString rawBytes = TestUtil.getAllExtensionsSet().toByteString(); + Message message = DynamicMessage.parseFrom( + TestAllExtensions.getDescriptor(), rawBytes, + TestUtil.getExtensionRegistry()); + extensionsReflectionTester.assertAllFieldsSetViaReflection(message); + + // Test Parser interface. + Message message2 = message.getParserForType().parseFrom( + rawBytes, TestUtil.getExtensionRegistry()); + extensionsReflectionTester.assertAllFieldsSetViaReflection(message2); } public void testDynamicMessagePackedSerialization() throws Exception { @@ -194,6 +228,10 @@ public class DynamicMessageTest extends TestCase { Message message2 = DynamicMessage.parseFrom(TestPackedTypes.getDescriptor(), rawBytes); packedReflectionTester.assertPackedFieldsSetViaReflection(message2); + + // Test Parser interface. + Message message3 = message2.getParserForType().parseFrom(rawBytes); + packedReflectionTester.assertPackedFieldsSetViaReflection(message3); } public void testDynamicMessageCopy() throws Exception { diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/GeneratedMessageTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/GeneratedMessageTest.java index acb2235..bf9db75 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/GeneratedMessageTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/GeneratedMessageTest.java @@ -30,6 +30,8 @@ package com.google.protobuf; +import com.google.protobuf.Descriptors.Descriptor; +import com.google.protobuf.Descriptors.FieldDescriptor; import com.google.protobuf.UnittestLite.TestAllExtensionsLite; import com.google.protobuf.test.UnittestImport; import protobuf_unittest.EnumWithNoOuter; @@ -53,6 +55,7 @@ import protobuf_unittest.UnittestProto.ForeignMessage; import protobuf_unittest.UnittestProto.ForeignMessageOrBuilder; import protobuf_unittest.UnittestProto.TestAllExtensions; import protobuf_unittest.UnittestProto.TestAllTypes; +import protobuf_unittest.UnittestProto.TestAllTypes.NestedMessage; import protobuf_unittest.UnittestProto.TestAllTypesOrBuilder; import protobuf_unittest.UnittestProto.TestExtremeDefaultValues; import protobuf_unittest.UnittestProto.TestPackedTypes; @@ -180,6 +183,33 @@ public class GeneratedMessageTest extends TestCase { assertIsUnmodifiable(value.getRepeatedFloatList()); } + public void testParsedMessagesAreImmutable() throws Exception { + TestAllTypes value = TestAllTypes.PARSER.parseFrom( + TestUtil.getAllSet().toByteString()); + assertIsUnmodifiable(value.getRepeatedInt32List()); + assertIsUnmodifiable(value.getRepeatedInt64List()); + assertIsUnmodifiable(value.getRepeatedUint32List()); + assertIsUnmodifiable(value.getRepeatedUint64List()); + assertIsUnmodifiable(value.getRepeatedSint32List()); + assertIsUnmodifiable(value.getRepeatedSint64List()); + assertIsUnmodifiable(value.getRepeatedFixed32List()); + assertIsUnmodifiable(value.getRepeatedFixed64List()); + assertIsUnmodifiable(value.getRepeatedSfixed32List()); + assertIsUnmodifiable(value.getRepeatedSfixed64List()); + assertIsUnmodifiable(value.getRepeatedFloatList()); + assertIsUnmodifiable(value.getRepeatedDoubleList()); + assertIsUnmodifiable(value.getRepeatedBoolList()); + assertIsUnmodifiable(value.getRepeatedStringList()); + assertIsUnmodifiable(value.getRepeatedBytesList()); + assertIsUnmodifiable(value.getRepeatedGroupList()); + assertIsUnmodifiable(value.getRepeatedNestedMessageList()); + assertIsUnmodifiable(value.getRepeatedForeignMessageList()); + assertIsUnmodifiable(value.getRepeatedImportMessageList()); + assertIsUnmodifiable(value.getRepeatedNestedEnumList()); + assertIsUnmodifiable(value.getRepeatedForeignEnumList()); + assertIsUnmodifiable(value.getRepeatedImportEnumList()); + } + private void assertIsUnmodifiable(List<?> list) { if (list == Collections.emptyList()) { // OKAY -- Need to check this b/c EmptyList allows you to call clear. @@ -881,7 +911,7 @@ public class GeneratedMessageTest extends TestCase { builder.setOptionalNestedMessage(nestedMessage1); assertEquals(3, mockParent.getInvalidationCount()); - // primitive repated + // primitive repeated builder.buildPartial(); builder.addRepeatedInt32(2); builder.addRepeatedInt32(3); @@ -977,4 +1007,140 @@ public class GeneratedMessageTest extends TestCase { assertSame(b1, messageOrBuilderList.get(1)); assertSame(m2, messageOrBuilderList.get(2)); } + + public void testGetFieldBuilder() { + Descriptor descriptor = TestAllTypes.getDescriptor(); + + FieldDescriptor fieldDescriptor = + descriptor.findFieldByName("optional_nested_message"); + FieldDescriptor foreignFieldDescriptor = + descriptor.findFieldByName("optional_foreign_message"); + FieldDescriptor importFieldDescriptor = + descriptor.findFieldByName("optional_import_message"); + + // Mutate the message with new field builder + // Mutate nested message + TestAllTypes.Builder builder1 = TestAllTypes.newBuilder(); + Message.Builder fieldBuilder1 = builder1.newBuilderForField(fieldDescriptor) + .mergeFrom((Message) builder1.getField(fieldDescriptor)); + FieldDescriptor subFieldDescriptor1 = + fieldBuilder1.getDescriptorForType().findFieldByName("bb"); + fieldBuilder1.setField(subFieldDescriptor1, 1); + builder1.setField(fieldDescriptor, fieldBuilder1.build()); + + // Mutate foreign message + Message.Builder foreignFieldBuilder1 = builder1.newBuilderForField( + foreignFieldDescriptor) + .mergeFrom((Message) builder1.getField(foreignFieldDescriptor)); + FieldDescriptor subForeignFieldDescriptor1 = + foreignFieldBuilder1.getDescriptorForType().findFieldByName("c"); + foreignFieldBuilder1.setField(subForeignFieldDescriptor1, 2); + builder1.setField(foreignFieldDescriptor, foreignFieldBuilder1.build()); + + // Mutate import message + Message.Builder importFieldBuilder1 = builder1.newBuilderForField( + importFieldDescriptor) + .mergeFrom((Message) builder1.getField(importFieldDescriptor)); + FieldDescriptor subImportFieldDescriptor1 = + importFieldBuilder1.getDescriptorForType().findFieldByName("d"); + importFieldBuilder1.setField(subImportFieldDescriptor1, 3); + builder1.setField(importFieldDescriptor, importFieldBuilder1.build()); + + Message newMessage1 = builder1.build(); + + // Mutate the message with existing field builder + // Mutate nested message + TestAllTypes.Builder builder2 = TestAllTypes.newBuilder(); + Message.Builder fieldBuilder2 = builder2.getFieldBuilder(fieldDescriptor); + FieldDescriptor subFieldDescriptor2 = + fieldBuilder2.getDescriptorForType().findFieldByName("bb"); + fieldBuilder2.setField(subFieldDescriptor2, 1); + builder2.setField(fieldDescriptor, fieldBuilder2.build()); + + // Mutate foreign message + Message.Builder foreignFieldBuilder2 = builder2.newBuilderForField( + foreignFieldDescriptor) + .mergeFrom((Message) builder2.getField(foreignFieldDescriptor)); + FieldDescriptor subForeignFieldDescriptor2 = + foreignFieldBuilder2.getDescriptorForType().findFieldByName("c"); + foreignFieldBuilder2.setField(subForeignFieldDescriptor2, 2); + builder2.setField(foreignFieldDescriptor, foreignFieldBuilder2.build()); + + // Mutate import message + Message.Builder importFieldBuilder2 = builder2.newBuilderForField( + importFieldDescriptor) + .mergeFrom((Message) builder2.getField(importFieldDescriptor)); + FieldDescriptor subImportFieldDescriptor2 = + importFieldBuilder2.getDescriptorForType().findFieldByName("d"); + importFieldBuilder2.setField(subImportFieldDescriptor2, 3); + builder2.setField(importFieldDescriptor, importFieldBuilder2.build()); + + Message newMessage2 = builder2.build(); + + // These two messages should be equal. + assertEquals(newMessage1, newMessage2); + } + + public void testGetFieldBuilderWithInitializedValue() { + Descriptor descriptor = TestAllTypes.getDescriptor(); + FieldDescriptor fieldDescriptor = + descriptor.findFieldByName("optional_nested_message"); + + // Before setting field, builder is initialized by default value. + TestAllTypes.Builder builder = TestAllTypes.newBuilder(); + NestedMessage.Builder fieldBuilder = + (NestedMessage.Builder) builder.getFieldBuilder(fieldDescriptor); + assertEquals(0, fieldBuilder.getBb()); + + // Setting field value with new field builder instance. + builder = TestAllTypes.newBuilder(); + NestedMessage.Builder newFieldBuilder = + builder.getOptionalNestedMessageBuilder(); + newFieldBuilder.setBb(2); + // Then get the field builder instance by getFieldBuilder(). + fieldBuilder = + (NestedMessage.Builder) builder.getFieldBuilder(fieldDescriptor); + // It should contain new value. + assertEquals(2, fieldBuilder.getBb()); + // These two builder should be equal. + assertSame(fieldBuilder, newFieldBuilder); + } + + public void testGetFieldBuilderNotSupportedException() { + Descriptor descriptor = TestAllTypes.getDescriptor(); + TestAllTypes.Builder builder = TestAllTypes.newBuilder(); + try { + builder.getFieldBuilder(descriptor.findFieldByName("optional_int32")); + fail("Exception was not thrown"); + } catch (UnsupportedOperationException e) { + // We expect this exception. + } + try { + builder.getFieldBuilder( + descriptor.findFieldByName("optional_nested_enum")); + fail("Exception was not thrown"); + } catch (UnsupportedOperationException e) { + // We expect this exception. + } + try { + builder.getFieldBuilder(descriptor.findFieldByName("repeated_int32")); + fail("Exception was not thrown"); + } catch (UnsupportedOperationException e) { + // We expect this exception. + } + try { + builder.getFieldBuilder( + descriptor.findFieldByName("repeated_nested_enum")); + fail("Exception was not thrown"); + } catch (UnsupportedOperationException e) { + // We expect this exception. + } + try { + builder.getFieldBuilder( + descriptor.findFieldByName("repeated_nested_message")); + fail("Exception was not thrown"); + } catch (UnsupportedOperationException e) { + // We expect this exception. + } + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8Test.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8Test.java new file mode 100644 index 0000000..b204b60 --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8Test.java @@ -0,0 +1,180 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import com.google.protobuf.IsValidUtf8TestUtil.Shard; + +import junit.framework.TestCase; + +import java.io.UnsupportedEncodingException; + +/** + * Tests cases for {@link ByteString#isValidUtf8()}. This includes three + * brute force tests that actually test every permutation of one byte, two byte, + * and three byte sequences to ensure that the method produces the right result + * for every possible byte encoding where "right" means it's consistent with + * java's UTF-8 string encoding/decoding such that the method returns true for + * any sequence that will round trip when converted to a String and then back to + * bytes and will return false for any sequence that will not round trip. + * See also {@link IsValidUtf8FourByteTest}. It also includes some + * other more targeted tests. + * + * @author jonp@google.com (Jon Perlow) + * @author martinrb@google.com (Martin Buchholz) + */ +public class IsValidUtf8Test extends TestCase { + + /** + * Tests that round tripping of all two byte permutations work. + */ + public void testIsValidUtf8_1Byte() throws UnsupportedEncodingException { + IsValidUtf8TestUtil.testBytes(1, + IsValidUtf8TestUtil.EXPECTED_ONE_BYTE_ROUNDTRIPPABLE_COUNT); + } + + /** + * Tests that round tripping of all two byte permutations work. + */ + public void testIsValidUtf8_2Bytes() throws UnsupportedEncodingException { + IsValidUtf8TestUtil.testBytes(2, + IsValidUtf8TestUtil.EXPECTED_TWO_BYTE_ROUNDTRIPPABLE_COUNT); + } + + /** + * Tests that round tripping of all three byte permutations work. + */ + public void testIsValidUtf8_3Bytes() throws UnsupportedEncodingException { + IsValidUtf8TestUtil.testBytes(3, + IsValidUtf8TestUtil.EXPECTED_THREE_BYTE_ROUNDTRIPPABLE_COUNT); + } + + /** + * Tests that round tripping of a sample of four byte permutations work. + * All permutations are prohibitively expensive to test for automated runs; + * {@link IsValidUtf8FourByteTest} is used for full coverage. This method + * tests specific four-byte cases. + */ + public void testIsValidUtf8_4BytesSamples() + throws UnsupportedEncodingException { + // Valid 4 byte. + assertValidUtf8(0xF0, 0xA4, 0xAD, 0xA2); + + // Bad trailing bytes + assertInvalidUtf8(0xF0, 0xA4, 0xAD, 0x7F); + assertInvalidUtf8(0xF0, 0xA4, 0xAD, 0xC0); + + // Special cases for byte2 + assertInvalidUtf8(0xF0, 0x8F, 0xAD, 0xA2); + assertInvalidUtf8(0xF4, 0x90, 0xAD, 0xA2); + } + + /** + * Tests some hard-coded test cases. + */ + public void testSomeSequences() { + // Empty + assertTrue(asBytes("").isValidUtf8()); + + // One-byte characters, including control characters + assertTrue(asBytes("\u0000abc\u007f").isValidUtf8()); + + // Two-byte characters + assertTrue(asBytes("\u00a2\u00a2").isValidUtf8()); + + // Three-byte characters + assertTrue(asBytes("\u020ac\u020ac").isValidUtf8()); + + // Four-byte characters + assertTrue(asBytes("\u024B62\u024B62").isValidUtf8()); + + // Mixed string + assertTrue( + asBytes("a\u020ac\u00a2b\\u024B62u020acc\u00a2de\u024B62") + .isValidUtf8()); + + // Not a valid string + assertInvalidUtf8(-1, 0, -1, 0); + } + + private byte[] toByteArray(int... bytes) { + byte[] realBytes = new byte[bytes.length]; + for (int i = 0; i < bytes.length; i++) { + realBytes[i] = (byte) bytes[i]; + } + return realBytes; + } + + private ByteString toByteString(int... bytes) { + return ByteString.copyFrom(toByteArray(bytes)); + } + + private void assertValidUtf8(int[] bytes, boolean not) { + byte[] realBytes = toByteArray(bytes); + assertTrue(not ^ Utf8.isValidUtf8(realBytes)); + assertTrue(not ^ Utf8.isValidUtf8(realBytes, 0, bytes.length)); + ByteString lit = ByteString.copyFrom(realBytes); + ByteString sub = lit.substring(0, bytes.length); + assertTrue(not ^ lit.isValidUtf8()); + assertTrue(not ^ sub.isValidUtf8()); + ByteString[] ropes = { + RopeByteString.newInstanceForTest(ByteString.EMPTY, lit), + RopeByteString.newInstanceForTest(ByteString.EMPTY, sub), + RopeByteString.newInstanceForTest(lit, ByteString.EMPTY), + RopeByteString.newInstanceForTest(sub, ByteString.EMPTY), + RopeByteString.newInstanceForTest(sub, lit) + }; + for (ByteString rope : ropes) { + assertTrue(not ^ rope.isValidUtf8()); + } + } + + private void assertValidUtf8(int... bytes) { + assertValidUtf8(bytes, false); + } + + private void assertInvalidUtf8(int... bytes) { + assertValidUtf8(bytes, true); + } + + private static ByteString asBytes(String s) { + return ByteString.copyFromUtf8(s); + } + + public void testShardsHaveExpectedRoundTrippables() { + // A sanity check. + int actual = 0; + for (Shard shard : IsValidUtf8TestUtil.FOUR_BYTE_SHARDS) { + actual += shard.expected; + } + assertEquals(IsValidUtf8TestUtil.EXPECTED_FOUR_BYTE_ROUNDTRIPPABLE_COUNT, + actual); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8TestUtil.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8TestUtil.java new file mode 100644 index 0000000..4cb3d5b --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/IsValidUtf8TestUtil.java @@ -0,0 +1,421 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import static junit.framework.Assert.*; + +import java.io.UnsupportedEncodingException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.Random; +import java.util.logging.Logger; +import java.nio.charset.CharsetDecoder; +import java.nio.charset.Charset; +import java.nio.charset.CodingErrorAction; +import java.nio.charset.CharsetEncoder; +import java.nio.charset.CoderResult; +import java.nio.ByteBuffer; +import java.nio.CharBuffer; + +/** + * Shared testing code for {@link IsValidUtf8Test} and + * {@link IsValidUtf8FourByteTest}. + * + * @author jonp@google.com (Jon Perlow) + * @author martinrb@google.com (Martin Buchholz) + */ +class IsValidUtf8TestUtil { + private static Logger logger = Logger.getLogger( + IsValidUtf8TestUtil.class.getName()); + + // 128 - [chars 0x0000 to 0x007f] + static long ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS = 0x007f - 0x0000 + 1; + + // 128 + static long EXPECTED_ONE_BYTE_ROUNDTRIPPABLE_COUNT = + ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS; + + // 1920 [chars 0x0080 to 0x07FF] + static long TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS = 0x07FF - 0x0080 + 1; + + // 18,304 + static long EXPECTED_TWO_BYTE_ROUNDTRIPPABLE_COUNT = + // Both bytes are one byte characters + (long) Math.pow(EXPECTED_ONE_BYTE_ROUNDTRIPPABLE_COUNT, 2) + + // The possible number of two byte characters + TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS; + + // 2048 + static long THREE_BYTE_SURROGATES = 2 * 1024; + + // 61,440 [chars 0x0800 to 0xFFFF, minus surrogates] + static long THREE_BYTE_ROUNDTRIPPABLE_CHARACTERS = + 0xFFFF - 0x0800 + 1 - THREE_BYTE_SURROGATES; + + // 2,650,112 + static long EXPECTED_THREE_BYTE_ROUNDTRIPPABLE_COUNT = + // All one byte characters + (long) Math.pow(EXPECTED_ONE_BYTE_ROUNDTRIPPABLE_COUNT, 3) + + // One two byte character and a one byte character + 2 * TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS * + ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS + + // Three byte characters + THREE_BYTE_ROUNDTRIPPABLE_CHARACTERS; + + // 1,048,576 [chars 0x10000L to 0x10FFFF] + static long FOUR_BYTE_ROUNDTRIPPABLE_CHARACTERS = 0x10FFFF - 0x10000L + 1; + + // 289,571,839 + static long EXPECTED_FOUR_BYTE_ROUNDTRIPPABLE_COUNT = + // All one byte characters + (long) Math.pow(EXPECTED_ONE_BYTE_ROUNDTRIPPABLE_COUNT, 4) + + // One and three byte characters + 2 * THREE_BYTE_ROUNDTRIPPABLE_CHARACTERS * + ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS + + // Two two byte characters + TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS * TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS + + // Permutations of one and two byte characters + 3 * TWO_BYTE_ROUNDTRIPPABLE_CHARACTERS * + ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS * + ONE_BYTE_ROUNDTRIPPABLE_CHARACTERS + + // Four byte characters + FOUR_BYTE_ROUNDTRIPPABLE_CHARACTERS; + + static class Shard { + final long index; + final long start; + final long lim; + final long expected; + + + public Shard(long index, long start, long lim, long expected) { + assertTrue(start < lim); + this.index = index; + this.start = start; + this.lim = lim; + this.expected = expected; + } + } + + static final long[] FOUR_BYTE_SHARDS_EXPECTED_ROUNTRIPPABLES = + generateFourByteShardsExpectedRunnables(); + + private static long[] generateFourByteShardsExpectedRunnables() { + long[] expected = new long[128]; + + // 0-63 are all 5300224 + for (int i = 0; i <= 63; i++) { + expected[i] = 5300224; + } + + // 97-111 are all 2342912 + for (int i = 97; i <= 111; i++) { + expected[i] = 2342912; + } + + // 113-117 are all 1048576 + for (int i = 113; i <= 117; i++) { + expected[i] = 1048576; + } + + // One offs + expected[112] = 786432; + expected[118] = 786432; + expected[119] = 1048576; + expected[120] = 458752; + expected[121] = 524288; + expected[122] = 65536; + + // Anything not assigned was the default 0. + return expected; + } + + static final List<Shard> FOUR_BYTE_SHARDS = generateFourByteShards( + 128, FOUR_BYTE_SHARDS_EXPECTED_ROUNTRIPPABLES); + + + private static List<Shard> generateFourByteShards( + int numShards, long[] expected) { + assertEquals(numShards, expected.length); + List<Shard> shards = new ArrayList<Shard>(numShards); + long LIM = 1L << 32; + long increment = LIM / numShards; + assertTrue(LIM % numShards == 0); + for (int i = 0; i < numShards; i++) { + shards.add(new Shard(i, + increment * i, + increment * (i + 1), + expected[i])); + } + return shards; + } + + /** + * Helper to run the loop to test all the permutations for the number of bytes + * specified. + * + * @param numBytes the number of bytes in the byte array + * @param expectedCount the expected number of roundtrippable permutations + */ + static void testBytes(int numBytes, long expectedCount) + throws UnsupportedEncodingException { + testBytes(numBytes, expectedCount, 0, -1); + } + + /** + * Helper to run the loop to test all the permutations for the number of bytes + * specified. This overload is useful for debugging to get the loop to start + * at a certain character. + * + * @param numBytes the number of bytes in the byte array + * @param expectedCount the expected number of roundtrippable permutations + * @param start the starting bytes encoded as a long as big-endian + * @param lim the limit of bytes to process encoded as a long as big-endian, + * or -1 to mean the max limit for numBytes + */ + static void testBytes(int numBytes, long expectedCount, long start, long lim) + throws UnsupportedEncodingException { + Random rnd = new Random(); + byte[] bytes = new byte[numBytes]; + + if (lim == -1) { + lim = 1L << (numBytes * 8); + } + long count = 0; + long countRoundTripped = 0; + for (long byteChar = start; byteChar < lim; byteChar++) { + long tmpByteChar = byteChar; + for (int i = 0; i < numBytes; i++) { + bytes[bytes.length - i - 1] = (byte) tmpByteChar; + tmpByteChar = tmpByteChar >> 8; + } + ByteString bs = ByteString.copyFrom(bytes); + boolean isRoundTrippable = bs.isValidUtf8(); + String s = new String(bytes, "UTF-8"); + byte[] bytesReencoded = s.getBytes("UTF-8"); + boolean bytesEqual = Arrays.equals(bytes, bytesReencoded); + + if (bytesEqual != isRoundTrippable) { + outputFailure(byteChar, bytes, bytesReencoded); + } + + // Check agreement with static Utf8 methods. + assertEquals(isRoundTrippable, Utf8.isValidUtf8(bytes)); + assertEquals(isRoundTrippable, Utf8.isValidUtf8(bytes, 0, numBytes)); + + // Test partial sequences. + // Partition numBytes into three segments (not necessarily non-empty). + int i = rnd.nextInt(numBytes); + int j = rnd.nextInt(numBytes); + if (j < i) { + int tmp = i; i = j; j = tmp; + } + int state1 = Utf8.partialIsValidUtf8(Utf8.COMPLETE, bytes, 0, i); + int state2 = Utf8.partialIsValidUtf8(state1, bytes, i, j); + int state3 = Utf8.partialIsValidUtf8(state2, bytes, j, numBytes); + if (isRoundTrippable != (state3 == Utf8.COMPLETE)) { + System.out.printf("state=%04x %04x %04x i=%d j=%d%n", + state1, state2, state3, i, j); + outputFailure(byteChar, bytes, bytesReencoded); + } + assertEquals(isRoundTrippable, (state3 == Utf8.COMPLETE)); + + // Test ropes built out of small partial sequences + ByteString rope = RopeByteString.newInstanceForTest( + bs.substring(0, i), + RopeByteString.newInstanceForTest( + bs.substring(i, j), + bs.substring(j, numBytes))); + assertSame(RopeByteString.class, rope.getClass()); + + ByteString[] byteStrings = { bs, bs.substring(0, numBytes), rope }; + for (ByteString x : byteStrings) { + assertEquals(isRoundTrippable, + x.isValidUtf8()); + assertEquals(state3, + x.partialIsValidUtf8(Utf8.COMPLETE, 0, numBytes)); + + assertEquals(state1, + x.partialIsValidUtf8(Utf8.COMPLETE, 0, i)); + assertEquals(state1, + x.substring(0, i).partialIsValidUtf8(Utf8.COMPLETE, 0, i)); + assertEquals(state2, + x.partialIsValidUtf8(state1, i, j - i)); + assertEquals(state2, + x.substring(i, j).partialIsValidUtf8(state1, 0, j - i)); + assertEquals(state3, + x.partialIsValidUtf8(state2, j, numBytes - j)); + assertEquals(state3, + x.substring(j, numBytes) + .partialIsValidUtf8(state2, 0, numBytes - j)); + } + + // ByteString reduplication should not affect its UTF-8 validity. + ByteString ropeADope = + RopeByteString.newInstanceForTest(bs, bs.substring(0, numBytes)); + assertEquals(isRoundTrippable, ropeADope.isValidUtf8()); + + if (isRoundTrippable) { + countRoundTripped++; + } + count++; + if (byteChar != 0 && byteChar % 1000000L == 0) { + logger.info("Processed " + (byteChar / 1000000L) + + " million characters"); + } + } + logger.info("Round tripped " + countRoundTripped + " of " + count); + assertEquals(expectedCount, countRoundTripped); + } + + /** + * Variation of {@link #testBytes} that does less allocation using the + * low-level encoders/decoders directly. Checked in because it's useful for + * debugging when trying to process bytes faster, but since it doesn't use the + * actual String class, it's possible for incompatibilities to develop + * (although unlikely). + * + * @param numBytes the number of bytes in the byte array + * @param expectedCount the expected number of roundtrippable permutations + * @param start the starting bytes encoded as a long as big-endian + * @param lim the limit of bytes to process encoded as a long as big-endian, + * or -1 to mean the max limit for numBytes + */ + void testBytesUsingByteBuffers( + int numBytes, long expectedCount, long start, long lim) + throws UnsupportedEncodingException { + CharsetDecoder decoder = Charset.forName("UTF-8").newDecoder() + .onMalformedInput(CodingErrorAction.REPLACE) + .onUnmappableCharacter(CodingErrorAction.REPLACE); + CharsetEncoder encoder = Charset.forName("UTF-8").newEncoder() + .onMalformedInput(CodingErrorAction.REPLACE) + .onUnmappableCharacter(CodingErrorAction.REPLACE); + byte[] bytes = new byte[numBytes]; + int maxChars = (int) (decoder.maxCharsPerByte() * numBytes) + 1; + char[] charsDecoded = + new char[(int) (decoder.maxCharsPerByte() * numBytes) + 1]; + int maxBytes = (int) (encoder.maxBytesPerChar() * maxChars) + 1; + byte[] bytesReencoded = new byte[maxBytes]; + + ByteBuffer bb = ByteBuffer.wrap(bytes); + CharBuffer cb = CharBuffer.wrap(charsDecoded); + ByteBuffer bbReencoded = ByteBuffer.wrap(bytesReencoded); + if (lim == -1) { + lim = 1L << (numBytes * 8); + } + long count = 0; + long countRoundTripped = 0; + for (long byteChar = start; byteChar < lim; byteChar++) { + bb.rewind(); + bb.limit(bytes.length); + cb.rewind(); + cb.limit(charsDecoded.length); + bbReencoded.rewind(); + bbReencoded.limit(bytesReencoded.length); + encoder.reset(); + decoder.reset(); + long tmpByteChar = byteChar; + for (int i = 0; i < bytes.length; i++) { + bytes[bytes.length - i - 1] = (byte) tmpByteChar; + tmpByteChar = tmpByteChar >> 8; + } + boolean isRoundTrippable = ByteString.copyFrom(bytes).isValidUtf8(); + CoderResult result = decoder.decode(bb, cb, true); + assertFalse(result.isError()); + result = decoder.flush(cb); + assertFalse(result.isError()); + + int charLen = cb.position(); + cb.rewind(); + cb.limit(charLen); + result = encoder.encode(cb, bbReencoded, true); + assertFalse(result.isError()); + result = encoder.flush(bbReencoded); + assertFalse(result.isError()); + + boolean bytesEqual = true; + int bytesLen = bbReencoded.position(); + if (bytesLen != numBytes) { + bytesEqual = false; + } else { + for (int i = 0; i < numBytes; i++) { + if (bytes[i] != bytesReencoded[i]) { + bytesEqual = false; + break; + } + } + } + if (bytesEqual != isRoundTrippable) { + outputFailure(byteChar, bytes, bytesReencoded, bytesLen); + } + + count++; + if (isRoundTrippable) { + countRoundTripped++; + } + if (byteChar != 0 && byteChar % 1000000 == 0) { + logger.info("Processed " + (byteChar / 1000000) + + " million characters"); + } + } + logger.info("Round tripped " + countRoundTripped + " of " + count); + assertEquals(expectedCount, countRoundTripped); + } + + private static void outputFailure(long byteChar, byte[] bytes, byte[] after) { + outputFailure(byteChar, bytes, after, after.length); + } + + private static void outputFailure(long byteChar, byte[] bytes, byte[] after, + int len) { + fail("Failure: (" + Long.toHexString(byteChar) + ") " + + toHexString(bytes) + " => " + toHexString(after, len)); + } + + private static String toHexString(byte[] b) { + return toHexString(b, b.length); + } + + private static String toHexString(byte[] b, int len) { + StringBuilder s = new StringBuilder(); + s.append("\""); + for (int i = 0; i < len; i++) { + if (i > 0) { + s.append(" "); + } + s.append(String.format("%02x", b[i] & 0xFF)); + } + s.append("\""); + return s.toString(); + } + +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringArrayListTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringArrayListTest.java index 4dcdc74..d500595fd 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringArrayListTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringArrayListTest.java @@ -32,6 +32,9 @@ package com.google.protobuf; import junit.framework.TestCase; +import java.util.ArrayList; +import java.util.List; + /** * Tests for {@link LazyStringArrayList}. * @@ -115,4 +118,45 @@ public class LazyStringArrayListTest extends TestCase { assertSame(aPrimeByteString, list.getByteString(0)); assertSame(bPrimeByteString, list.getByteString(1)); } + + public void testCopyConstructorCopiesByReference() { + LazyStringArrayList list1 = new LazyStringArrayList(); + list1.add(STRING_A); + list1.add(BYTE_STRING_B); + list1.add(BYTE_STRING_C); + + LazyStringArrayList list2 = new LazyStringArrayList(list1); + assertEquals(3, list2.size()); + assertSame(STRING_A, list2.get(0)); + assertSame(BYTE_STRING_B, list2.getByteString(1)); + assertSame(BYTE_STRING_C, list2.getByteString(2)); + } + + public void testListCopyConstructor() { + List<String> list1 = new ArrayList<String>(); + list1.add(STRING_A); + list1.add(STRING_B); + list1.add(STRING_C); + + LazyStringArrayList list2 = new LazyStringArrayList(list1); + assertEquals(3, list2.size()); + assertSame(STRING_A, list2.get(0)); + assertSame(STRING_B, list2.get(1)); + assertSame(STRING_C, list2.get(2)); + } + + public void testAddAllCopiesByReferenceIfPossible() { + LazyStringArrayList list1 = new LazyStringArrayList(); + list1.add(STRING_A); + list1.add(BYTE_STRING_B); + list1.add(BYTE_STRING_C); + + LazyStringArrayList list2 = new LazyStringArrayList(); + list2.addAll(list1); + + assertEquals(3, list2.size()); + assertSame(STRING_A, list2.get(0)); + assertSame(BYTE_STRING_B, list2.getByteString(1)); + assertSame(BYTE_STRING_C, list2.getByteString(2)); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringEndToEndTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringEndToEndTest.java index e6870b5..fe9599e 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringEndToEndTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/LazyStringEndToEndTest.java @@ -50,6 +50,19 @@ public class LazyStringEndToEndTest extends TestCase { 114, 4, -1, 0, -1, 0, -30, 2, 4, -1, 0, -1, 0, -30, 2, 4, -1, 0, -1, 0, }); + private ByteString encodedTestAllTypes; + + @Override + protected void setUp() throws Exception { + super.setUp(); + this.encodedTestAllTypes = UnittestProto.TestAllTypes.newBuilder() + .setOptionalString("foo") + .addRepeatedString("bar") + .addRepeatedString("baz") + .build() + .toByteString(); + } + /** * Tests that an invalid UTF8 string will roundtrip through a parse * and serialization. @@ -112,4 +125,19 @@ public class LazyStringEndToEndTest extends TestCase { assertSame(bPrime, proto.getRepeatedString(0)); assertSame(cPrime, proto.getRepeatedString(1)); } + + public void testNoStringCachingIfOnlyBytesAccessed() throws Exception { + UnittestProto.TestAllTypes proto = + UnittestProto.TestAllTypes.parseFrom(encodedTestAllTypes); + ByteString optional = proto.getOptionalStringBytes(); + assertSame(optional, proto.getOptionalStringBytes()); + assertSame(optional, proto.toBuilder().getOptionalStringBytes()); + + ByteString repeated0 = proto.getRepeatedStringBytes(0); + ByteString repeated1 = proto.getRepeatedStringBytes(1); + assertSame(repeated0, proto.getRepeatedStringBytes(0)); + assertSame(repeated1, proto.getRepeatedStringBytes(1)); + assertSame(repeated0, proto.toBuilder().getRepeatedStringBytes(0)); + assertSame(repeated1, proto.toBuilder().getRepeatedStringBytes(1)); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/LiteralByteStringTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/LiteralByteStringTest.java new file mode 100644 index 0000000..deee1ee --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/LiteralByteStringTest.java @@ -0,0 +1,396 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import junit.framework.TestCase; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.UnsupportedEncodingException; +import java.nio.ByteBuffer; +import java.util.Arrays; +import java.util.List; +import java.util.NoSuchElementException; + +/** + * Test {@link LiteralByteString} by setting up a reference string in {@link #setUp()}. + * This class is designed to be extended for testing extensions of {@link LiteralByteString} + * such as {@link BoundedByteString}, see {@link BoundedByteStringTest}. + * + * @author carlanton@google.com (Carl Haverl) + */ +public class LiteralByteStringTest extends TestCase { + protected static final String UTF_8 = "UTF-8"; + + protected String classUnderTest; + protected byte[] referenceBytes; + protected ByteString stringUnderTest; + protected int expectedHashCode; + + @Override + protected void setUp() throws Exception { + classUnderTest = "LiteralByteString"; + referenceBytes = ByteStringTest.getTestBytes(1234, 11337766L); + stringUnderTest = ByteString.copyFrom(referenceBytes); + expectedHashCode = 331161852; + } + + public void testExpectedType() { + String actualClassName = getActualClassName(stringUnderTest); + assertEquals(classUnderTest + " should match type exactly", classUnderTest, actualClassName); + } + + protected String getActualClassName(Object object) { + String actualClassName = object.getClass().getName(); + actualClassName = actualClassName.substring(actualClassName.lastIndexOf('.') + 1); + return actualClassName; + } + + public void testByteAt() { + boolean stillEqual = true; + for (int i = 0; stillEqual && i < referenceBytes.length; ++i) { + stillEqual = (referenceBytes[i] == stringUnderTest.byteAt(i)); + } + assertTrue(classUnderTest + " must capture the right bytes", stillEqual); + } + + public void testByteIterator() { + boolean stillEqual = true; + ByteString.ByteIterator iter = stringUnderTest.iterator(); + for (int i = 0; stillEqual && i < referenceBytes.length; ++i) { + stillEqual = (iter.hasNext() && referenceBytes[i] == iter.nextByte()); + } + assertTrue(classUnderTest + " must capture the right bytes", stillEqual); + assertFalse(classUnderTest + " must have exhausted the itertor", iter.hasNext()); + + try { + iter.nextByte(); + fail("Should have thrown an exception."); + } catch (NoSuchElementException e) { + // This is success + } + } + + public void testByteIterable() { + boolean stillEqual = true; + int j = 0; + for (byte quantum : stringUnderTest) { + stillEqual = (referenceBytes[j] == quantum); + ++j; + } + assertTrue(classUnderTest + " must capture the right bytes as Bytes", stillEqual); + assertEquals(classUnderTest + " iterable character count", referenceBytes.length, j); + } + + public void testSize() { + assertEquals(classUnderTest + " must have the expected size", referenceBytes.length, + stringUnderTest.size()); + } + + public void testGetTreeDepth() { + assertEquals(classUnderTest + " must have depth 0", 0, stringUnderTest.getTreeDepth()); + } + + public void testIsBalanced() { + assertTrue(classUnderTest + " is technically balanced", stringUnderTest.isBalanced()); + } + + public void testCopyTo_ByteArrayOffsetLength() { + int destinationOffset = 50; + int length = 100; + byte[] destination = new byte[destinationOffset + length]; + int sourceOffset = 213; + stringUnderTest.copyTo(destination, sourceOffset, destinationOffset, length); + boolean stillEqual = true; + for (int i = 0; stillEqual && i < length; ++i) { + stillEqual = referenceBytes[i + sourceOffset] == destination[i + destinationOffset]; + } + assertTrue(classUnderTest + ".copyTo(4 arg) must give the expected bytes", stillEqual); + } + + public void testCopyTo_ByteArrayOffsetLengthErrors() { + int destinationOffset = 50; + int length = 100; + byte[] destination = new byte[destinationOffset + length]; + + try { + // Copy one too many bytes + stringUnderTest.copyTo(destination, stringUnderTest.size() + 1 - length, + destinationOffset, length); + fail("Should have thrown an exception when copying too many bytes of a " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + + try { + // Copy with illegal negative sourceOffset + stringUnderTest.copyTo(destination, -1, destinationOffset, length); + fail("Should have thrown an exception when given a negative sourceOffset in " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + + try { + // Copy with illegal negative destinationOffset + stringUnderTest.copyTo(destination, 0, -1, length); + fail("Should have thrown an exception when given a negative destinationOffset in " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + + try { + // Copy with illegal negative size + stringUnderTest.copyTo(destination, 0, 0, -1); + fail("Should have thrown an exception when given a negative size in " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + + try { + // Copy with illegal too-large sourceOffset + stringUnderTest.copyTo(destination, 2 * stringUnderTest.size(), 0, length); + fail("Should have thrown an exception when the destinationOffset is too large in " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + + try { + // Copy with illegal too-large destinationOffset + stringUnderTest.copyTo(destination, 0, 2 * destination.length, length); + fail("Should have thrown an exception when the destinationOffset is too large in " + + classUnderTest); + } catch (IndexOutOfBoundsException expected) { + // This is success + } + } + + public void testCopyTo_ByteBuffer() { + ByteBuffer myBuffer = ByteBuffer.allocate(referenceBytes.length); + stringUnderTest.copyTo(myBuffer); + assertTrue(classUnderTest + ".copyTo(ByteBuffer) must give back the same bytes", + Arrays.equals(referenceBytes, myBuffer.array())); + } + + public void testAsReadOnlyByteBuffer() { + ByteBuffer byteBuffer = stringUnderTest.asReadOnlyByteBuffer(); + byte[] roundTripBytes = new byte[referenceBytes.length]; + assertTrue(byteBuffer.remaining() == referenceBytes.length); + assertTrue(byteBuffer.isReadOnly()); + byteBuffer.get(roundTripBytes); + assertTrue(classUnderTest + ".asReadOnlyByteBuffer() must give back the same bytes", + Arrays.equals(referenceBytes, roundTripBytes)); + } + + public void testAsReadOnlyByteBufferList() { + List<ByteBuffer> byteBuffers = stringUnderTest.asReadOnlyByteBufferList(); + int bytesSeen = 0; + byte[] roundTripBytes = new byte[referenceBytes.length]; + for (ByteBuffer byteBuffer : byteBuffers) { + int thisLength = byteBuffer.remaining(); + assertTrue(byteBuffer.isReadOnly()); + assertTrue(bytesSeen + thisLength <= referenceBytes.length); + byteBuffer.get(roundTripBytes, bytesSeen, thisLength); + bytesSeen += thisLength; + } + assertTrue(bytesSeen == referenceBytes.length); + assertTrue(classUnderTest + ".asReadOnlyByteBufferTest() must give back the same bytes", + Arrays.equals(referenceBytes, roundTripBytes)); + } + + public void testToByteArray() { + byte[] roundTripBytes = stringUnderTest.toByteArray(); + assertTrue(classUnderTest + ".toByteArray() must give back the same bytes", + Arrays.equals(referenceBytes, roundTripBytes)); + } + + public void testWriteTo() throws IOException { + ByteArrayOutputStream bos = new ByteArrayOutputStream(); + stringUnderTest.writeTo(bos); + byte[] roundTripBytes = bos.toByteArray(); + assertTrue(classUnderTest + ".writeTo() must give back the same bytes", + Arrays.equals(referenceBytes, roundTripBytes)); + } + + public void testWriteTo_mutating() throws IOException { + OutputStream os = new OutputStream() { + @Override + public void write(byte[] b, int off, int len) { + for (int x = 0; x < len; ++x) { + b[off + x] = (byte) 0; + } + } + + @Override + public void write(int b) { + // Purposefully left blank. + } + }; + + stringUnderTest.writeTo(os); + byte[] newBytes = stringUnderTest.toByteArray(); + assertTrue(classUnderTest + ".writeTo() must not grant access to underlying array", + Arrays.equals(referenceBytes, newBytes)); + } + + public void testNewOutput() throws IOException { + ByteArrayOutputStream bos = new ByteArrayOutputStream(); + ByteString.Output output = ByteString.newOutput(); + stringUnderTest.writeTo(output); + assertEquals("Output Size returns correct result", + output.size(), stringUnderTest.size()); + output.writeTo(bos); + assertTrue("Output.writeTo() must give back the same bytes", + Arrays.equals(referenceBytes, bos.toByteArray())); + + // write the output stream to itself! This should cause it to double + output.writeTo(output); + assertEquals("Writing an output stream to itself is successful", + stringUnderTest.concat(stringUnderTest), output.toByteString()); + + output.reset(); + assertEquals("Output.reset() resets the output", 0, output.size()); + assertEquals("Output.reset() resets the output", + ByteString.EMPTY, output.toByteString()); + + } + + public void testToString() throws UnsupportedEncodingException { + String testString = "I love unicode \u1234\u5678 characters"; + LiteralByteString unicode = new LiteralByteString(testString.getBytes(UTF_8)); + String roundTripString = unicode.toString(UTF_8); + assertEquals(classUnderTest + " unicode must match", testString, roundTripString); + } + + public void testEquals() { + assertEquals(classUnderTest + " must not equal null", false, stringUnderTest.equals(null)); + assertEquals(classUnderTest + " must equal self", stringUnderTest, stringUnderTest); + assertFalse(classUnderTest + " must not equal the empty string", + stringUnderTest.equals(ByteString.EMPTY)); + assertEquals(classUnderTest + " empty strings must be equal", + new LiteralByteString(new byte[]{}), stringUnderTest.substring(55, 55)); + assertEquals(classUnderTest + " must equal another string with the same value", + stringUnderTest, new LiteralByteString(referenceBytes)); + + byte[] mungedBytes = new byte[referenceBytes.length]; + System.arraycopy(referenceBytes, 0, mungedBytes, 0, referenceBytes.length); + mungedBytes[mungedBytes.length - 5] ^= 0xFF; + assertFalse(classUnderTest + " must not equal every string with the same length", + stringUnderTest.equals(new LiteralByteString(mungedBytes))); + } + + public void testHashCode() { + int hash = stringUnderTest.hashCode(); + assertEquals(classUnderTest + " must have expected hashCode", expectedHashCode, hash); + } + + public void testPeekCachedHashCode() { + assertEquals(classUnderTest + ".peekCachedHashCode() should return zero at first", 0, + stringUnderTest.peekCachedHashCode()); + stringUnderTest.hashCode(); + assertEquals(classUnderTest + ".peekCachedHashCode should return zero at first", + expectedHashCode, stringUnderTest.peekCachedHashCode()); + } + + public void testPartialHash() { + // partialHash() is more strenuously tested elsewhere by testing hashes of substrings. + // This test would fail if the expected hash were 1. It's not. + int hash = stringUnderTest.partialHash(stringUnderTest.size(), 0, stringUnderTest.size()); + assertEquals(classUnderTest + ".partialHash() must yield expected hashCode", + expectedHashCode, hash); + } + + public void testNewInput() throws IOException { + InputStream input = stringUnderTest.newInput(); + assertEquals("InputStream.available() returns correct value", + stringUnderTest.size(), input.available()); + boolean stillEqual = true; + for (byte referenceByte : referenceBytes) { + int expectedInt = (referenceByte & 0xFF); + stillEqual = (expectedInt == input.read()); + } + assertEquals("InputStream.available() returns correct value", + 0, input.available()); + assertTrue(classUnderTest + " must give the same bytes from the InputStream", stillEqual); + assertEquals(classUnderTest + " InputStream must now be exhausted", -1, input.read()); + } + + public void testNewInput_skip() throws IOException { + InputStream input = stringUnderTest.newInput(); + int stringSize = stringUnderTest.size(); + int nearEndIndex = stringSize * 2 / 3; + long skipped1 = input.skip(nearEndIndex); + assertEquals("InputStream.skip()", skipped1, nearEndIndex); + assertEquals("InputStream.available()", + stringSize - skipped1, input.available()); + assertTrue("InputStream.mark() is available", input.markSupported()); + input.mark(0); + assertEquals("InputStream.skip(), read()", + stringUnderTest.byteAt(nearEndIndex) & 0xFF, input.read()); + assertEquals("InputStream.available()", + stringSize - skipped1 - 1, input.available()); + long skipped2 = input.skip(stringSize); + assertEquals("InputStream.skip() incomplete", + skipped2, stringSize - skipped1 - 1); + assertEquals("InputStream.skip(), no more input", 0, input.available()); + assertEquals("InputStream.skip(), no more input", -1, input.read()); + input.reset(); + assertEquals("InputStream.reset() succeded", + stringSize - skipped1, input.available()); + assertEquals("InputStream.reset(), read()", + stringUnderTest.byteAt(nearEndIndex) & 0xFF, input.read()); + } + + public void testNewCodedInput() throws IOException { + CodedInputStream cis = stringUnderTest.newCodedInput(); + byte[] roundTripBytes = cis.readRawBytes(referenceBytes.length); + assertTrue(classUnderTest + " must give the same bytes back from the CodedInputStream", + Arrays.equals(referenceBytes, roundTripBytes)); + assertTrue(classUnderTest + " CodedInputStream must now be exhausted", cis.isAtEnd()); + } + + /** + * Make sure we keep things simple when concatenating with empty. See also + * {@link ByteStringTest#testConcat_empty()}. + */ + public void testConcat_empty() { + assertSame(classUnderTest + " concatenated with empty must give " + classUnderTest, + stringUnderTest.concat(ByteString.EMPTY), stringUnderTest); + assertSame("empty concatenated with " + classUnderTest + " must give " + classUnderTest, + ByteString.EMPTY.concat(stringUnderTest), stringUnderTest); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/MessageTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/MessageTest.java index c2f47eb..747fed7 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/MessageTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/MessageTest.java @@ -38,6 +38,8 @@ import protobuf_unittest.UnittestProto.ForeignMessage; import junit.framework.TestCase; +import java.util.List; + /** * Misc. unit tests for message operations that apply to both generated * and dynamic messages. @@ -310,4 +312,42 @@ public class MessageTest extends TestCase { assertEquals("Message missing required fields: a, b, c", e.getMessage()); } } + + /** Test reading unset repeated message from DynamicMessage. */ + public void testDynamicRepeatedMessageNull() throws Exception { + Descriptors.Descriptor descriptor = TestRequired.getDescriptor(); + DynamicMessage result = + DynamicMessage.newBuilder(TestAllTypes.getDescriptor()) + .mergeFrom(DynamicMessage.newBuilder(MERGE_SOURCE).build()) + .build(); + + assertTrue(result.getField(result.getDescriptorForType() + .findFieldByName("repeated_foreign_message")) instanceof List<?>); + assertEquals(result.getRepeatedFieldCount(result.getDescriptorForType() + .findFieldByName("repeated_foreign_message")), 0); + } + + /** Test reading repeated message from DynamicMessage. */ + public void testDynamicRepeatedMessageNotNull() throws Exception { + + TestAllTypes REPEATED_NESTED = + TestAllTypes.newBuilder() + .setOptionalInt32(1) + .setOptionalString("foo") + .setOptionalForeignMessage(ForeignMessage.getDefaultInstance()) + .addRepeatedString("bar") + .addRepeatedForeignMessage(ForeignMessage.getDefaultInstance()) + .addRepeatedForeignMessage(ForeignMessage.getDefaultInstance()) + .build(); + Descriptors.Descriptor descriptor = TestRequired.getDescriptor(); + DynamicMessage result = + DynamicMessage.newBuilder(TestAllTypes.getDescriptor()) + .mergeFrom(DynamicMessage.newBuilder(REPEATED_NESTED).build()) + .build(); + + assertTrue(result.getField(result.getDescriptorForType() + .findFieldByName("repeated_foreign_message")) instanceof List<?>); + assertEquals(result.getRepeatedFieldCount(result.getDescriptorForType() + .findFieldByName("repeated_foreign_message")), 2); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/ParserTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/ParserTest.java new file mode 100644 index 0000000..396902c --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/ParserTest.java @@ -0,0 +1,375 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import com.google.protobuf.UnittestLite.TestAllTypesLite; +import com.google.protobuf.UnittestLite.TestPackedExtensionsLite; +import com.google.protobuf.UnittestLite.TestParsingMergeLite; +import com.google.protobuf.UnittestLite; +import protobuf_unittest.UnittestOptimizeFor.TestOptimizedForSize; +import protobuf_unittest.UnittestOptimizeFor.TestRequiredOptimizedForSize; +import protobuf_unittest.UnittestOptimizeFor; +import protobuf_unittest.UnittestProto.ForeignMessage; +import protobuf_unittest.UnittestProto.TestAllTypes; +import protobuf_unittest.UnittestProto.TestEmptyMessage; +import protobuf_unittest.UnittestProto.TestRequired; +import protobuf_unittest.UnittestProto.TestParsingMerge; +import protobuf_unittest.UnittestProto; + +import junit.framework.TestCase; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; + +/** + * Unit test for {@link Parser}. + * + * @author liujisi@google.com (Pherl Liu) + */ +public class ParserTest extends TestCase { + public void testGeneratedMessageParserSingleton() throws Exception { + for (int i = 0; i < 10; i++) { + assertEquals(TestAllTypes.PARSER, + TestUtil.getAllSet().getParserForType()); + } + } + + private void assertRoundTripEquals(MessageLite message, + ExtensionRegistryLite registry) + throws Exception { + final byte[] data = message.toByteArray(); + final int offset = 20; + final int length = data.length; + final int padding = 30; + Parser<? extends MessageLite> parser = message.getParserForType(); + assertMessageEquals(message, parser.parseFrom(data, registry)); + assertMessageEquals(message, parser.parseFrom( + generatePaddingArray(data, offset, padding), + offset, length, registry)); + assertMessageEquals(message, parser.parseFrom( + message.toByteString(), registry)); + assertMessageEquals(message, parser.parseFrom( + new ByteArrayInputStream(data), registry)); + assertMessageEquals(message, parser.parseFrom( + CodedInputStream.newInstance(data), registry)); + } + + private void assertRoundTripEquals(MessageLite message) throws Exception { + final byte[] data = message.toByteArray(); + final int offset = 20; + final int length = data.length; + final int padding = 30; + Parser<? extends MessageLite> parser = message.getParserForType(); + assertMessageEquals(message, parser.parseFrom(data)); + assertMessageEquals(message, parser.parseFrom( + generatePaddingArray(data, offset, padding), + offset, length)); + assertMessageEquals(message, parser.parseFrom(message.toByteString())); + assertMessageEquals(message, parser.parseFrom( + new ByteArrayInputStream(data))); + assertMessageEquals(message, parser.parseFrom( + CodedInputStream.newInstance(data))); + } + + private void assertMessageEquals(MessageLite expected, MessageLite actual) + throws Exception { + if (expected instanceof Message) { + assertEquals(expected, actual); + } else { + assertEquals(expected.toByteString(), actual.toByteString()); + } + } + + private byte[] generatePaddingArray(byte[] data, int offset, int padding) { + byte[] result = new byte[offset + data.length + padding]; + System.arraycopy(data, 0, result, offset, data.length); + return result; + } + + public void testNormalMessage() throws Exception { + assertRoundTripEquals(TestUtil.getAllSet()); + } + + public void testParsePartial() throws Exception { + Parser<TestRequired> parser = TestRequired.PARSER; + final String errorString = + "Should throw exceptions when the parsed message isn't initialized."; + + // TestRequired.b and TestRequired.c are not set. + TestRequired partialMessage = TestRequired.newBuilder() + .setA(1).buildPartial(); + + // parsePartialFrom should pass. + byte[] data = partialMessage.toByteArray(); + assertEquals(partialMessage, parser.parsePartialFrom(data)); + assertEquals(partialMessage, parser.parsePartialFrom( + partialMessage.toByteString())); + assertEquals(partialMessage, parser.parsePartialFrom( + new ByteArrayInputStream(data))); + assertEquals(partialMessage, parser.parsePartialFrom( + CodedInputStream.newInstance(data))); + + // parseFrom(ByteArray) + try { + parser.parseFrom(partialMessage.toByteArray()); + fail(errorString); + } catch (InvalidProtocolBufferException e) { + // pass. + } + + // parseFrom(ByteString) + try { + parser.parseFrom(partialMessage.toByteString()); + fail(errorString); + } catch (InvalidProtocolBufferException e) { + // pass. + } + + // parseFrom(InputStream) + try { + parser.parseFrom(new ByteArrayInputStream(partialMessage.toByteArray())); + fail(errorString); + } catch (IOException e) { + // pass. + } + + // parseFrom(CodedInputStream) + try { + parser.parseFrom(CodedInputStream.newInstance( + partialMessage.toByteArray())); + fail(errorString); + } catch (IOException e) { + // pass. + } + } + + public void testParseExtensions() throws Exception { + assertRoundTripEquals(TestUtil.getAllExtensionsSet(), + TestUtil.getExtensionRegistry()); + assertRoundTripEquals(TestUtil.getAllLiteExtensionsSet(), + TestUtil.getExtensionRegistryLite()); + } + + public void testParsePacked() throws Exception { + assertRoundTripEquals(TestUtil.getPackedSet()); + assertRoundTripEquals(TestUtil.getPackedExtensionsSet(), + TestUtil.getExtensionRegistry()); + assertRoundTripEquals(TestUtil.getLitePackedExtensionsSet(), + TestUtil.getExtensionRegistryLite()); + } + + public void testParseDelimitedTo() throws Exception { + // Write normal Message. + TestAllTypes normalMessage = TestUtil.getAllSet(); + ByteArrayOutputStream output = new ByteArrayOutputStream(); + normalMessage.writeDelimitedTo(output); + + // Write MessageLite with packed extension fields. + TestPackedExtensionsLite packedMessage = + TestUtil.getLitePackedExtensionsSet(); + packedMessage.writeDelimitedTo(output); + + InputStream input = new ByteArrayInputStream(output.toByteArray()); + assertMessageEquals( + normalMessage, + normalMessage.getParserForType().parseDelimitedFrom(input)); + assertMessageEquals( + packedMessage, + packedMessage.getParserForType().parseDelimitedFrom( + input, TestUtil.getExtensionRegistryLite())); + } + + public void testParseUnknownFields() throws Exception { + // All fields will be treated as unknown fields in emptyMessage. + TestEmptyMessage emptyMessage = TestEmptyMessage.PARSER.parseFrom( + TestUtil.getAllSet().toByteString()); + assertEquals( + TestUtil.getAllSet().toByteString(), + emptyMessage.toByteString()); + } + + public void testOptimizeForSize() throws Exception { + TestOptimizedForSize.Builder builder = TestOptimizedForSize.newBuilder(); + builder.setI(12).setMsg(ForeignMessage.newBuilder().setC(34).build()); + builder.setExtension(TestOptimizedForSize.testExtension, 56); + builder.setExtension(TestOptimizedForSize.testExtension2, + TestRequiredOptimizedForSize.newBuilder().setX(78).build()); + + TestOptimizedForSize message = builder.build(); + ExtensionRegistry registry = ExtensionRegistry.newInstance(); + UnittestOptimizeFor.registerAllExtensions(registry); + + assertRoundTripEquals(message, registry); + } + + /** Helper method for {@link #testParsingMerge()}.*/ + private void assertMessageMerged(TestAllTypes allTypes) + throws Exception { + assertEquals(3, allTypes.getOptionalInt32()); + assertEquals(2, allTypes.getOptionalInt64()); + assertEquals("hello", allTypes.getOptionalString()); + } + + /** Helper method for {@link #testParsingMergeLite()}.*/ + private void assertMessageMerged(TestAllTypesLite allTypes) + throws Exception { + assertEquals(3, allTypes.getOptionalInt32()); + assertEquals(2, allTypes.getOptionalInt64()); + assertEquals("hello", allTypes.getOptionalString()); + } + + public void testParsingMerge() throws Exception { + // Build messages. + TestAllTypes.Builder builder = TestAllTypes.newBuilder(); + TestAllTypes msg1 = builder.setOptionalInt32(1).build(); + builder.clear(); + TestAllTypes msg2 = builder.setOptionalInt64(2).build(); + builder.clear(); + TestAllTypes msg3 = builder.setOptionalInt32(3) + .setOptionalString("hello").build(); + + // Build groups. + TestParsingMerge.RepeatedFieldsGenerator.Group1 optionalG1 = + TestParsingMerge.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg1).build(); + TestParsingMerge.RepeatedFieldsGenerator.Group1 optionalG2 = + TestParsingMerge.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg2).build(); + TestParsingMerge.RepeatedFieldsGenerator.Group1 optionalG3 = + TestParsingMerge.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg3).build(); + TestParsingMerge.RepeatedFieldsGenerator.Group2 repeatedG1 = + TestParsingMerge.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg1).build(); + TestParsingMerge.RepeatedFieldsGenerator.Group2 repeatedG2 = + TestParsingMerge.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg2).build(); + TestParsingMerge.RepeatedFieldsGenerator.Group2 repeatedG3 = + TestParsingMerge.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg3).build(); + + // Assign and serialize RepeatedFieldsGenerator. + ByteString data = TestParsingMerge.RepeatedFieldsGenerator.newBuilder() + .addField1(msg1).addField1(msg2).addField1(msg3) + .addField2(msg1).addField2(msg2).addField2(msg3) + .addField3(msg1).addField3(msg2).addField3(msg3) + .addGroup1(optionalG1).addGroup1(optionalG2).addGroup1(optionalG3) + .addGroup2(repeatedG1).addGroup2(repeatedG2).addGroup2(repeatedG3) + .addExt1(msg1).addExt1(msg2).addExt1(msg3) + .addExt2(msg1).addExt2(msg2).addExt2(msg3) + .build().toByteString(); + + // Parse TestParsingMerge. + ExtensionRegistry registry = ExtensionRegistry.newInstance(); + UnittestProto.registerAllExtensions(registry); + TestParsingMerge parsingMerge = + TestParsingMerge.PARSER.parseFrom(data, registry); + + // Required and optional fields should be merged. + assertMessageMerged(parsingMerge.getRequiredAllTypes()); + assertMessageMerged(parsingMerge.getOptionalAllTypes()); + assertMessageMerged( + parsingMerge.getOptionalGroup().getOptionalGroupAllTypes()); + assertMessageMerged(parsingMerge.getExtension( + TestParsingMerge.optionalExt)); + + // Repeated fields should not be merged. + assertEquals(3, parsingMerge.getRepeatedAllTypesCount()); + assertEquals(3, parsingMerge.getRepeatedGroupCount()); + assertEquals(3, parsingMerge.getExtensionCount( + TestParsingMerge.repeatedExt)); + } + + public void testParsingMergeLite() throws Exception { + // Build messages. + TestAllTypesLite.Builder builder = + TestAllTypesLite.newBuilder(); + TestAllTypesLite msg1 = builder.setOptionalInt32(1).build(); + builder.clear(); + TestAllTypesLite msg2 = builder.setOptionalInt64(2).build(); + builder.clear(); + TestAllTypesLite msg3 = builder.setOptionalInt32(3) + .setOptionalString("hello").build(); + + // Build groups. + TestParsingMergeLite.RepeatedFieldsGenerator.Group1 optionalG1 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg1).build(); + TestParsingMergeLite.RepeatedFieldsGenerator.Group1 optionalG2 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg2).build(); + TestParsingMergeLite.RepeatedFieldsGenerator.Group1 optionalG3 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group1.newBuilder() + .setField1(msg3).build(); + TestParsingMergeLite.RepeatedFieldsGenerator.Group2 repeatedG1 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg1).build(); + TestParsingMergeLite.RepeatedFieldsGenerator.Group2 repeatedG2 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg2).build(); + TestParsingMergeLite.RepeatedFieldsGenerator.Group2 repeatedG3 = + TestParsingMergeLite.RepeatedFieldsGenerator.Group2.newBuilder() + .setField1(msg3).build(); + + // Assign and serialize RepeatedFieldsGenerator. + ByteString data = TestParsingMergeLite.RepeatedFieldsGenerator.newBuilder() + .addField1(msg1).addField1(msg2).addField1(msg3) + .addField2(msg1).addField2(msg2).addField2(msg3) + .addField3(msg1).addField3(msg2).addField3(msg3) + .addGroup1(optionalG1).addGroup1(optionalG2).addGroup1(optionalG3) + .addGroup2(repeatedG1).addGroup2(repeatedG2).addGroup2(repeatedG3) + .addExt1(msg1).addExt1(msg2).addExt1(msg3) + .addExt2(msg1).addExt2(msg2).addExt2(msg3) + .build().toByteString(); + + // Parse TestParsingMergeLite. + ExtensionRegistry registry = ExtensionRegistry.newInstance(); + UnittestLite.registerAllExtensions(registry); + TestParsingMergeLite parsingMerge = + TestParsingMergeLite.PARSER.parseFrom(data, registry); + + // Required and optional fields should be merged. + assertMessageMerged(parsingMerge.getRequiredAllTypes()); + assertMessageMerged(parsingMerge.getOptionalAllTypes()); + assertMessageMerged( + parsingMerge.getOptionalGroup().getOptionalGroupAllTypes()); + assertMessageMerged(parsingMerge.getExtension( + TestParsingMergeLite.optionalExt)); + + // Repeated fields should not be merged. + assertEquals(3, parsingMerge.getRepeatedAllTypesCount()); + assertEquals(3, parsingMerge.getRepeatedGroupCount()); + assertEquals(3, parsingMerge.getExtensionCount( + TestParsingMergeLite.repeatedExt)); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringSubstringTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringSubstringTest.java new file mode 100644 index 0000000..06707a2 --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringSubstringTest.java @@ -0,0 +1,97 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.UnsupportedEncodingException; +import java.util.Iterator; + +/** + * This class tests {@link RopeByteString#substring(int, int)} by inheriting the tests from + * {@link LiteralByteStringTest}. Only a couple of methods are overridden. + * + * @author carlanton@google.com (Carl Haverl) + */ +public class RopeByteStringSubstringTest extends LiteralByteStringTest { + + @Override + protected void setUp() throws Exception { + classUnderTest = "RopeByteString"; + byte[] sourceBytes = ByteStringTest.getTestBytes(22341, 22337766L); + Iterator<ByteString> iter = ByteStringTest.makeConcretePieces(sourceBytes).iterator(); + ByteString sourceString = iter.next(); + while (iter.hasNext()) { + sourceString = sourceString.concat(iter.next()); + } + + int from = 1130; + int to = sourceBytes.length - 5555; + stringUnderTest = sourceString.substring(from, to); + referenceBytes = new byte[to - from]; + System.arraycopy(sourceBytes, from, referenceBytes, 0, to - from); + expectedHashCode = -1259260680; + } + + @Override + public void testGetTreeDepth() { + assertEquals(classUnderTest + " must have the expected tree depth", + 3, stringUnderTest.getTreeDepth()); + } + + @Override + public void testToString() throws UnsupportedEncodingException { + String sourceString = "I love unicode \u1234\u5678 characters"; + ByteString sourceByteString = ByteString.copyFromUtf8(sourceString); + int copies = 250; + + // By building the RopeByteString by concatenating, this is actually a fairly strenuous test. + StringBuilder builder = new StringBuilder(copies * sourceString.length()); + ByteString unicode = ByteString.EMPTY; + for (int i = 0; i < copies; ++i) { + builder.append(sourceString); + unicode = RopeByteString.concatenate(unicode, sourceByteString); + } + String testString = builder.toString(); + + // Do the substring part + testString = testString.substring(2, testString.length() - 6); + unicode = unicode.substring(2, unicode.size() - 6); + + assertEquals(classUnderTest + " from string must have the expected type", + classUnderTest, getActualClassName(unicode)); + String roundTripString = unicode.toString(UTF_8); + assertEquals(classUnderTest + " unicode bytes must match", + testString, roundTripString); + ByteString flatString = ByteString.copyFromUtf8(testString); + assertEquals(classUnderTest + " string must equal the flat string", flatString, unicode); + assertEquals(classUnderTest + " string must must have same hashCode as the flat string", + flatString.hashCode(), unicode.hashCode()); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringTest.java new file mode 100644 index 0000000..15f660d --- /dev/null +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/RopeByteStringTest.java @@ -0,0 +1,115 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +package com.google.protobuf; + +import java.io.UnsupportedEncodingException; +import java.util.Arrays; +import java.util.Iterator; + +/** + * This class tests {@link RopeByteString} by inheriting the tests from + * {@link LiteralByteStringTest}. Only a couple of methods are overridden. + * + * <p>A full test of the result of {@link RopeByteString#substring(int, int)} is found in the + * separate class {@link RopeByteStringSubstringTest}. + * + * @author carlanton@google.com (Carl Haverl) + */ +public class RopeByteStringTest extends LiteralByteStringTest { + + @Override + protected void setUp() throws Exception { + classUnderTest = "RopeByteString"; + referenceBytes = ByteStringTest.getTestBytes(22341, 22337766L); + Iterator<ByteString> iter = ByteStringTest.makeConcretePieces(referenceBytes).iterator(); + stringUnderTest = iter.next(); + while (iter.hasNext()) { + stringUnderTest = stringUnderTest.concat(iter.next()); + } + expectedHashCode = -1214197238; + } + + @Override + public void testGetTreeDepth() { + assertEquals(classUnderTest + " must have the expected tree depth", + 4, stringUnderTest.getTreeDepth()); + } + + public void testBalance() { + int numberOfPieces = 10000; + int pieceSize = 64; + byte[] testBytes = ByteStringTest.getTestBytes(numberOfPieces * pieceSize, 113377L); + + // Build up a big ByteString from smaller pieces to force a rebalance + ByteString concatenated = ByteString.EMPTY; + for (int i = 0; i < numberOfPieces; ++i) { + concatenated = concatenated.concat(ByteString.copyFrom(testBytes, i * pieceSize, pieceSize)); + } + + assertEquals(classUnderTest + " from string must have the expected type", + classUnderTest, getActualClassName(concatenated)); + assertTrue(classUnderTest + " underlying bytes must match after balancing", + Arrays.equals(testBytes, concatenated.toByteArray())); + ByteString testString = ByteString.copyFrom(testBytes); + assertTrue(classUnderTest + " balanced string must equal flat string", + concatenated.equals(testString)); + assertTrue(classUnderTest + " flat string must equal balanced string", + testString.equals(concatenated)); + assertEquals(classUnderTest + " balanced string must have same hash code as flat string", + testString.hashCode(), concatenated.hashCode()); + } + + @Override + public void testToString() throws UnsupportedEncodingException { + String sourceString = "I love unicode \u1234\u5678 characters"; + ByteString sourceByteString = ByteString.copyFromUtf8(sourceString); + int copies = 250; + + // By building the RopeByteString by concatenating, this is actually a fairly strenuous test. + StringBuilder builder = new StringBuilder(copies * sourceString.length()); + ByteString unicode = ByteString.EMPTY; + for (int i = 0; i < copies; ++i) { + builder.append(sourceString); + unicode = RopeByteString.concatenate(unicode, sourceByteString); + } + String testString = builder.toString(); + + assertEquals(classUnderTest + " from string must have the expected type", + classUnderTest, getActualClassName(unicode)); + String roundTripString = unicode.toString(UTF_8); + assertEquals(classUnderTest + " unicode bytes must match", + testString, roundTripString); + ByteString flatString = ByteString.copyFromUtf8(testString); + assertEquals(classUnderTest + " string must equal the flat string", flatString, unicode); + assertEquals(classUnderTest + " string must must have same hashCode as the flat string", + flatString.hashCode(), unicode.hashCode()); + } +} diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/TestBadIdentifiers.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/TestBadIdentifiers.java index 6feec4e6..382acf0 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/TestBadIdentifiers.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/TestBadIdentifiers.java @@ -35,7 +35,7 @@ import junit.framework.TestCase; /** * Tests that proto2 api generation doesn't cause compile errors when * compiling protocol buffers that have names that would otherwise conflict - * if not fully qualified (like @Deprecated and @Override). + * if not fully qualified (like @Deprecated and @Override). * * @author jonp@google.com (Jon Perlow) */ @@ -45,5 +45,19 @@ public class TestBadIdentifiers extends TestCase { // If this compiles, it means the generation was correct. TestBadIdentifiersProto.Deprecated.newBuilder(); TestBadIdentifiersProto.Override.newBuilder(); - } + } + + public void testGetDescriptor() { + Descriptors.FileDescriptor fileDescriptor = + TestBadIdentifiersProto.getDescriptor(); + String descriptorField = TestBadIdentifiersProto.Descriptor + .getDefaultInstance().getDescriptor(); + Descriptors.Descriptor protoDescriptor = TestBadIdentifiersProto.Descriptor + .getDefaultInstance().getDescriptorForType(); + String nestedDescriptorField = TestBadIdentifiersProto.Descriptor + .NestedDescriptor.getDefaultInstance().getDescriptor(); + Descriptors.Descriptor nestedProtoDescriptor = TestBadIdentifiersProto + .Descriptor.NestedDescriptor.getDefaultInstance() + .getDescriptorForType(); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/TestUtil.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/TestUtil.java index 9109f41..76f5c60 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/TestUtil.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/TestUtil.java @@ -72,14 +72,16 @@ import static protobuf_unittest.UnittestProto.optionalBoolExtension; import static protobuf_unittest.UnittestProto.optionalStringExtension; import static protobuf_unittest.UnittestProto.optionalBytesExtension; import static protobuf_unittest.UnittestProto.optionalGroupExtension; -import static protobuf_unittest.UnittestProto.optionalNestedMessageExtension; +import static protobuf_unittest.UnittestProto.optionalCordExtension; +import static protobuf_unittest.UnittestProto.optionalForeignEnumExtension; import static protobuf_unittest.UnittestProto.optionalForeignMessageExtension; +import static protobuf_unittest.UnittestProto.optionalImportEnumExtension; import static protobuf_unittest.UnittestProto.optionalImportMessageExtension; import static protobuf_unittest.UnittestProto.optionalNestedEnumExtension; -import static protobuf_unittest.UnittestProto.optionalForeignEnumExtension; -import static protobuf_unittest.UnittestProto.optionalImportEnumExtension; +import static protobuf_unittest.UnittestProto.optionalNestedMessageExtension; +import static protobuf_unittest.UnittestProto.optionalPublicImportMessageExtension; +import static protobuf_unittest.UnittestProto.optionalLazyMessageExtension; import static protobuf_unittest.UnittestProto.optionalStringPieceExtension; -import static protobuf_unittest.UnittestProto.optionalCordExtension; import static protobuf_unittest.UnittestProto.repeatedInt32Extension; import static protobuf_unittest.UnittestProto.repeatedInt64Extension; @@ -100,6 +102,7 @@ import static protobuf_unittest.UnittestProto.repeatedGroupExtension; import static protobuf_unittest.UnittestProto.repeatedNestedMessageExtension; import static protobuf_unittest.UnittestProto.repeatedForeignMessageExtension; import static protobuf_unittest.UnittestProto.repeatedImportMessageExtension; +import static protobuf_unittest.UnittestProto.repeatedLazyMessageExtension; import static protobuf_unittest.UnittestProto.repeatedNestedEnumExtension; import static protobuf_unittest.UnittestProto.repeatedForeignEnumExtension; import static protobuf_unittest.UnittestProto.repeatedImportEnumExtension; @@ -162,11 +165,13 @@ import static com.google.protobuf.UnittestLite.optionalStringExtensionLite; import static com.google.protobuf.UnittestLite.optionalBytesExtensionLite; import static com.google.protobuf.UnittestLite.optionalGroupExtensionLite; import static com.google.protobuf.UnittestLite.optionalNestedMessageExtensionLite; +import static com.google.protobuf.UnittestLite.optionalForeignEnumExtensionLite; import static com.google.protobuf.UnittestLite.optionalForeignMessageExtensionLite; +import static com.google.protobuf.UnittestLite.optionalImportEnumExtensionLite; import static com.google.protobuf.UnittestLite.optionalImportMessageExtensionLite; import static com.google.protobuf.UnittestLite.optionalNestedEnumExtensionLite; -import static com.google.protobuf.UnittestLite.optionalForeignEnumExtensionLite; -import static com.google.protobuf.UnittestLite.optionalImportEnumExtensionLite; +import static com.google.protobuf.UnittestLite.optionalPublicImportMessageExtensionLite; +import static com.google.protobuf.UnittestLite.optionalLazyMessageExtensionLite; import static com.google.protobuf.UnittestLite.optionalStringPieceExtensionLite; import static com.google.protobuf.UnittestLite.optionalCordExtensionLite; @@ -189,6 +194,7 @@ import static com.google.protobuf.UnittestLite.repeatedGroupExtensionLite; import static com.google.protobuf.UnittestLite.repeatedNestedMessageExtensionLite; import static com.google.protobuf.UnittestLite.repeatedForeignMessageExtensionLite; import static com.google.protobuf.UnittestLite.repeatedImportMessageExtensionLite; +import static com.google.protobuf.UnittestLite.repeatedLazyMessageExtensionLite; import static com.google.protobuf.UnittestLite.repeatedNestedEnumExtensionLite; import static com.google.protobuf.UnittestLite.repeatedForeignEnumExtensionLite; import static com.google.protobuf.UnittestLite.repeatedImportEnumExtensionLite; @@ -222,8 +228,9 @@ import protobuf_unittest.UnittestProto.TestPackedTypes; import protobuf_unittest.UnittestProto.TestUnpackedTypes; import protobuf_unittest.UnittestProto.ForeignMessage; import protobuf_unittest.UnittestProto.ForeignEnum; -import com.google.protobuf.test.UnittestImport.ImportMessage; import com.google.protobuf.test.UnittestImport.ImportEnum; +import com.google.protobuf.test.UnittestImport.ImportMessage; +import com.google.protobuf.test.UnittestImportPublic.PublicImportMessage; import com.google.protobuf.UnittestLite.TestAllTypesLite; import com.google.protobuf.UnittestLite.TestAllExtensionsLite; @@ -231,8 +238,9 @@ import com.google.protobuf.UnittestLite.TestAllExtensionsLiteOrBuilder; import com.google.protobuf.UnittestLite.TestPackedExtensionsLite; import com.google.protobuf.UnittestLite.ForeignMessageLite; import com.google.protobuf.UnittestLite.ForeignEnumLite; -import com.google.protobuf.UnittestImportLite.ImportMessageLite; import com.google.protobuf.UnittestImportLite.ImportEnumLite; +import com.google.protobuf.UnittestImportLite.ImportMessageLite; +import com.google.protobuf.UnittestImportPublicLite.PublicImportMessageLite; import junit.framework.Assert; @@ -242,7 +250,7 @@ import java.io.RandomAccessFile; /** * Contains methods for setting all fields of {@code TestAllTypes} to - * some vaules as well as checking that all the fields are set to those values. + * some values as well as checking that all the fields are set to those values. * These are useful for testing various protocol message features, e.g. * set all fields of a message, serialize it, parse it, and check that all * fields are set. @@ -275,6 +283,16 @@ public final class TestUtil { } /** + * Get a {@code TestAllTypes.Builder} with all fields set as they would be by + * {@link #setAllFields(TestAllTypes.Builder)}. + */ + public static TestAllTypes.Builder getAllSetBuilder() { + TestAllTypes.Builder builder = TestAllTypes.newBuilder(); + setAllFields(builder); + return builder; + } + + /** * Get a {@code TestAllExtensions} with all fields set as they would be by * {@link #setAllExtensions(TestAllExtensions.Builder)}. */ @@ -344,6 +362,10 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(119).build()); message.setOptionalImportMessage( ImportMessage.newBuilder().setD(120).build()); + message.setOptionalPublicImportMessage( + PublicImportMessage.newBuilder().setE(126).build()); + message.setOptionalLazyMessage( + TestAllTypes.NestedMessage.newBuilder().setBb(127).build()); message.setOptionalNestedEnum (TestAllTypes.NestedEnum.BAZ); message.setOptionalForeignEnum(ForeignEnum.FOREIGN_BAZ); @@ -378,6 +400,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(219).build()); message.addRepeatedImportMessage( ImportMessage.newBuilder().setD(220).build()); + message.addRepeatedLazyMessage( + TestAllTypes.NestedMessage.newBuilder().setBb(227).build()); message.addRepeatedNestedEnum (TestAllTypes.NestedEnum.BAR); message.addRepeatedForeignEnum(ForeignEnum.FOREIGN_BAR); @@ -411,6 +435,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(319).build()); message.addRepeatedImportMessage( ImportMessage.newBuilder().setD(320).build()); + message.addRepeatedLazyMessage( + TestAllTypes.NestedMessage.newBuilder().setBb(327).build()); message.addRepeatedNestedEnum (TestAllTypes.NestedEnum.BAZ); message.addRepeatedForeignEnum(ForeignEnum.FOREIGN_BAZ); @@ -476,6 +502,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(519).build()); message.setRepeatedImportMessage(1, ImportMessage.newBuilder().setD(520).build()); + message.setRepeatedLazyMessage(1, + TestAllTypes.NestedMessage.newBuilder().setBb(527).build()); message.setRepeatedNestedEnum (1, TestAllTypes.NestedEnum.FOO); message.setRepeatedForeignEnum(1, ForeignEnum.FOREIGN_FOO); @@ -541,10 +569,12 @@ public final class TestUtil { Assert.assertEquals("115", message.getOptionalString ()); Assert.assertEquals(toBytes("116"), message.getOptionalBytes()); - Assert.assertEquals(117, message.getOptionalGroup ().getA()); - Assert.assertEquals(118, message.getOptionalNestedMessage ().getBb()); - Assert.assertEquals(119, message.getOptionalForeignMessage().getC()); - Assert.assertEquals(120, message.getOptionalImportMessage ().getD()); + Assert.assertEquals(117, message.getOptionalGroup ().getA()); + Assert.assertEquals(118, message.getOptionalNestedMessage ().getBb()); + Assert.assertEquals(119, message.getOptionalForeignMessage ().getC()); + Assert.assertEquals(120, message.getOptionalImportMessage ().getD()); + Assert.assertEquals(126, message.getOptionalPublicImportMessage().getE()); + Assert.assertEquals(127, message.getOptionalLazyMessage ().getBb()); Assert.assertEquals(TestAllTypes.NestedEnum.BAZ, message.getOptionalNestedEnum()); Assert.assertEquals(ForeignEnum.FOREIGN_BAZ, message.getOptionalForeignEnum()); @@ -575,6 +605,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getRepeatedNestedMessageCount ()); Assert.assertEquals(2, message.getRepeatedForeignMessageCount()); Assert.assertEquals(2, message.getRepeatedImportMessageCount ()); + Assert.assertEquals(2, message.getRepeatedLazyMessageCount ()); Assert.assertEquals(2, message.getRepeatedNestedEnumCount ()); Assert.assertEquals(2, message.getRepeatedForeignEnumCount ()); Assert.assertEquals(2, message.getRepeatedImportEnumCount ()); @@ -602,6 +633,7 @@ public final class TestUtil { Assert.assertEquals(218, message.getRepeatedNestedMessage (0).getBb()); Assert.assertEquals(219, message.getRepeatedForeignMessage(0).getC()); Assert.assertEquals(220, message.getRepeatedImportMessage (0).getD()); + Assert.assertEquals(227, message.getRepeatedLazyMessage (0).getBb()); Assert.assertEquals(TestAllTypes.NestedEnum.BAR, message.getRepeatedNestedEnum (0)); Assert.assertEquals(ForeignEnum.FOREIGN_BAR, message.getRepeatedForeignEnum(0)); @@ -630,6 +662,7 @@ public final class TestUtil { Assert.assertEquals(318, message.getRepeatedNestedMessage (1).getBb()); Assert.assertEquals(319, message.getRepeatedForeignMessage(1).getC()); Assert.assertEquals(320, message.getRepeatedImportMessage (1).getD()); + Assert.assertEquals(327, message.getRepeatedLazyMessage (1).getBb()); Assert.assertEquals(TestAllTypes.NestedEnum.BAZ, message.getRepeatedNestedEnum (1)); Assert.assertEquals(ForeignEnum.FOREIGN_BAZ, message.getRepeatedForeignEnum(1)); @@ -741,15 +774,19 @@ public final class TestUtil { Assert.assertEquals(ByteString.EMPTY, message.getOptionalBytes()); // Embedded messages should also be clear. - Assert.assertFalse(message.getOptionalGroup ().hasA()); - Assert.assertFalse(message.getOptionalNestedMessage ().hasBb()); - Assert.assertFalse(message.getOptionalForeignMessage().hasC()); - Assert.assertFalse(message.getOptionalImportMessage ().hasD()); - - Assert.assertEquals(0, message.getOptionalGroup ().getA()); - Assert.assertEquals(0, message.getOptionalNestedMessage ().getBb()); - Assert.assertEquals(0, message.getOptionalForeignMessage().getC()); - Assert.assertEquals(0, message.getOptionalImportMessage ().getD()); + Assert.assertFalse(message.getOptionalGroup ().hasA()); + Assert.assertFalse(message.getOptionalNestedMessage ().hasBb()); + Assert.assertFalse(message.getOptionalForeignMessage ().hasC()); + Assert.assertFalse(message.getOptionalImportMessage ().hasD()); + Assert.assertFalse(message.getOptionalPublicImportMessage().hasE()); + Assert.assertFalse(message.getOptionalLazyMessage ().hasBb()); + + Assert.assertEquals(0, message.getOptionalGroup ().getA()); + Assert.assertEquals(0, message.getOptionalNestedMessage ().getBb()); + Assert.assertEquals(0, message.getOptionalForeignMessage ().getC()); + Assert.assertEquals(0, message.getOptionalImportMessage ().getD()); + Assert.assertEquals(0, message.getOptionalPublicImportMessage().getE()); + Assert.assertEquals(0, message.getOptionalLazyMessage ().getBb()); // Enums without defaults are set to the first value in the enum. Assert.assertEquals(TestAllTypes.NestedEnum.FOO, message.getOptionalNestedEnum ()); @@ -780,6 +817,7 @@ public final class TestUtil { Assert.assertEquals(0, message.getRepeatedNestedMessageCount ()); Assert.assertEquals(0, message.getRepeatedForeignMessageCount()); Assert.assertEquals(0, message.getRepeatedImportMessageCount ()); + Assert.assertEquals(0, message.getRepeatedLazyMessageCount ()); Assert.assertEquals(0, message.getRepeatedNestedEnumCount ()); Assert.assertEquals(0, message.getRepeatedForeignEnumCount ()); Assert.assertEquals(0, message.getRepeatedImportEnumCount ()); @@ -868,6 +906,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getRepeatedNestedMessageCount ()); Assert.assertEquals(2, message.getRepeatedForeignMessageCount()); Assert.assertEquals(2, message.getRepeatedImportMessageCount ()); + Assert.assertEquals(2, message.getRepeatedLazyMessageCount ()); Assert.assertEquals(2, message.getRepeatedNestedEnumCount ()); Assert.assertEquals(2, message.getRepeatedForeignEnumCount ()); Assert.assertEquals(2, message.getRepeatedImportEnumCount ()); @@ -895,6 +934,7 @@ public final class TestUtil { Assert.assertEquals(218, message.getRepeatedNestedMessage (0).getBb()); Assert.assertEquals(219, message.getRepeatedForeignMessage(0).getC()); Assert.assertEquals(220, message.getRepeatedImportMessage (0).getD()); + Assert.assertEquals(227, message.getRepeatedLazyMessage (0).getBb()); Assert.assertEquals(TestAllTypes.NestedEnum.BAR, message.getRepeatedNestedEnum (0)); Assert.assertEquals(ForeignEnum.FOREIGN_BAR, message.getRepeatedForeignEnum(0)); @@ -924,6 +964,7 @@ public final class TestUtil { Assert.assertEquals(518, message.getRepeatedNestedMessage (1).getBb()); Assert.assertEquals(519, message.getRepeatedForeignMessage(1).getC()); Assert.assertEquals(520, message.getRepeatedImportMessage (1).getD()); + Assert.assertEquals(527, message.getRepeatedLazyMessage (1).getBb()); Assert.assertEquals(TestAllTypes.NestedEnum.FOO, message.getRepeatedNestedEnum (1)); Assert.assertEquals(ForeignEnum.FOREIGN_FOO, message.getRepeatedForeignEnum(1)); @@ -1210,6 +1251,10 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(119).build()); message.setExtension(optionalImportMessageExtension, ImportMessage.newBuilder().setD(120).build()); + message.setExtension(optionalPublicImportMessageExtension, + PublicImportMessage.newBuilder().setE(126).build()); + message.setExtension(optionalLazyMessageExtension, + TestAllTypes.NestedMessage.newBuilder().setBb(127).build()); message.setExtension(optionalNestedEnumExtension, TestAllTypes.NestedEnum.BAZ); message.setExtension(optionalForeignEnumExtension, ForeignEnum.FOREIGN_BAZ); @@ -1244,6 +1289,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(219).build()); message.addExtension(repeatedImportMessageExtension, ImportMessage.newBuilder().setD(220).build()); + message.addExtension(repeatedLazyMessageExtension, + TestAllTypes.NestedMessage.newBuilder().setBb(227).build()); message.addExtension(repeatedNestedEnumExtension, TestAllTypes.NestedEnum.BAR); message.addExtension(repeatedForeignEnumExtension, ForeignEnum.FOREIGN_BAR); @@ -1277,6 +1324,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(319).build()); message.addExtension(repeatedImportMessageExtension, ImportMessage.newBuilder().setD(320).build()); + message.addExtension(repeatedLazyMessageExtension, + TestAllTypes.NestedMessage.newBuilder().setBb(327).build()); message.addExtension(repeatedNestedEnumExtension, TestAllTypes.NestedEnum.BAZ); message.addExtension(repeatedForeignEnumExtension, ForeignEnum.FOREIGN_BAZ); @@ -1343,6 +1392,8 @@ public final class TestUtil { ForeignMessage.newBuilder().setC(519).build()); message.setExtension(repeatedImportMessageExtension, 1, ImportMessage.newBuilder().setD(520).build()); + message.setExtension(repeatedLazyMessageExtension, 1, + TestAllTypes.NestedMessage.newBuilder().setBb(527).build()); message.setExtension(repeatedNestedEnumExtension , 1, TestAllTypes.NestedEnum.FOO); message.setExtension(repeatedForeignEnumExtension, 1, ForeignEnum.FOREIGN_FOO); @@ -1409,10 +1460,12 @@ public final class TestUtil { assertEqualsExactType("115", message.getExtension(optionalStringExtension )); assertEqualsExactType(toBytes("116"), message.getExtension(optionalBytesExtension)); - assertEqualsExactType(117, message.getExtension(optionalGroupExtension ).getA()); - assertEqualsExactType(118, message.getExtension(optionalNestedMessageExtension ).getBb()); - assertEqualsExactType(119, message.getExtension(optionalForeignMessageExtension).getC()); - assertEqualsExactType(120, message.getExtension(optionalImportMessageExtension ).getD()); + assertEqualsExactType(117, message.getExtension(optionalGroupExtension ).getA()); + assertEqualsExactType(118, message.getExtension(optionalNestedMessageExtension ).getBb()); + assertEqualsExactType(119, message.getExtension(optionalForeignMessageExtension ).getC()); + assertEqualsExactType(120, message.getExtension(optionalImportMessageExtension ).getD()); + assertEqualsExactType(126, message.getExtension(optionalPublicImportMessageExtension).getE()); + assertEqualsExactType(127, message.getExtension(optionalLazyMessageExtension ).getBb()); assertEqualsExactType(TestAllTypes.NestedEnum.BAZ, message.getExtension(optionalNestedEnumExtension)); @@ -1446,6 +1499,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getExtensionCount(repeatedNestedMessageExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignMessageExtension)); Assert.assertEquals(2, message.getExtensionCount(repeatedImportMessageExtension )); + Assert.assertEquals(2, message.getExtensionCount(repeatedLazyMessageExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedNestedEnumExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignEnumExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedImportEnumExtension )); @@ -1473,6 +1527,7 @@ public final class TestUtil { assertEqualsExactType(218, message.getExtension(repeatedNestedMessageExtension , 0).getBb()); assertEqualsExactType(219, message.getExtension(repeatedForeignMessageExtension, 0).getC()); assertEqualsExactType(220, message.getExtension(repeatedImportMessageExtension , 0).getD()); + assertEqualsExactType(227, message.getExtension(repeatedLazyMessageExtension , 0).getBb()); assertEqualsExactType(TestAllTypes.NestedEnum.BAR, message.getExtension(repeatedNestedEnumExtension, 0)); @@ -1504,6 +1559,7 @@ public final class TestUtil { assertEqualsExactType(318, message.getExtension(repeatedNestedMessageExtension , 1).getBb()); assertEqualsExactType(319, message.getExtension(repeatedForeignMessageExtension, 1).getC()); assertEqualsExactType(320, message.getExtension(repeatedImportMessageExtension , 1).getD()); + assertEqualsExactType(327, message.getExtension(repeatedLazyMessageExtension , 1).getBb()); assertEqualsExactType(TestAllTypes.NestedEnum.BAZ, message.getExtension(repeatedNestedEnumExtension, 1)); @@ -1664,6 +1720,7 @@ public final class TestUtil { Assert.assertEquals(0, message.getExtensionCount(repeatedNestedMessageExtension )); Assert.assertEquals(0, message.getExtensionCount(repeatedForeignMessageExtension)); Assert.assertEquals(0, message.getExtensionCount(repeatedImportMessageExtension )); + Assert.assertEquals(0, message.getExtensionCount(repeatedLazyMessageExtension )); Assert.assertEquals(0, message.getExtensionCount(repeatedNestedEnumExtension )); Assert.assertEquals(0, message.getExtensionCount(repeatedForeignEnumExtension )); Assert.assertEquals(0, message.getExtensionCount(repeatedImportEnumExtension )); @@ -1692,6 +1749,7 @@ public final class TestUtil { Assert.assertEquals(0, message.getExtension(repeatedNestedMessageExtension ).size()); Assert.assertEquals(0, message.getExtension(repeatedForeignMessageExtension).size()); Assert.assertEquals(0, message.getExtension(repeatedImportMessageExtension ).size()); + Assert.assertEquals(0, message.getExtension(repeatedLazyMessageExtension ).size()); Assert.assertEquals(0, message.getExtension(repeatedNestedEnumExtension ).size()); Assert.assertEquals(0, message.getExtension(repeatedForeignEnumExtension ).size()); Assert.assertEquals(0, message.getExtension(repeatedImportEnumExtension ).size()); @@ -1783,6 +1841,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getExtensionCount(repeatedNestedMessageExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignMessageExtension)); Assert.assertEquals(2, message.getExtensionCount(repeatedImportMessageExtension )); + Assert.assertEquals(2, message.getExtensionCount(repeatedLazyMessageExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedNestedEnumExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignEnumExtension )); Assert.assertEquals(2, message.getExtensionCount(repeatedImportEnumExtension )); @@ -1810,6 +1869,7 @@ public final class TestUtil { assertEqualsExactType(218, message.getExtension(repeatedNestedMessageExtension , 0).getBb()); assertEqualsExactType(219, message.getExtension(repeatedForeignMessageExtension, 0).getC()); assertEqualsExactType(220, message.getExtension(repeatedImportMessageExtension , 0).getD()); + assertEqualsExactType(227, message.getExtension(repeatedLazyMessageExtension , 0).getBb()); assertEqualsExactType(TestAllTypes.NestedEnum.BAR, message.getExtension(repeatedNestedEnumExtension, 0)); @@ -1842,6 +1902,7 @@ public final class TestUtil { assertEqualsExactType(518, message.getExtension(repeatedNestedMessageExtension , 1).getBb()); assertEqualsExactType(519, message.getExtension(repeatedForeignMessageExtension, 1).getC()); assertEqualsExactType(520, message.getExtension(repeatedImportMessageExtension , 1).getD()); + assertEqualsExactType(527, message.getExtension(repeatedLazyMessageExtension , 1).getBb()); assertEqualsExactType(TestAllTypes.NestedEnum.FOO, message.getExtension(repeatedNestedEnumExtension, 1)); @@ -1965,6 +2026,10 @@ public final class TestUtil { ForeignMessageLite.newBuilder().setC(119).build()); message.setExtension(optionalImportMessageExtensionLite, ImportMessageLite.newBuilder().setD(120).build()); + message.setExtension(optionalPublicImportMessageExtensionLite, + PublicImportMessageLite.newBuilder().setE(126).build()); + message.setExtension(optionalLazyMessageExtensionLite, + TestAllTypesLite.NestedMessage.newBuilder().setBb(127).build()); message.setExtension(optionalNestedEnumExtensionLite, TestAllTypesLite.NestedEnum.BAZ); message.setExtension(optionalForeignEnumExtensionLite, ForeignEnumLite.FOREIGN_LITE_BAZ); @@ -1999,6 +2064,8 @@ public final class TestUtil { ForeignMessageLite.newBuilder().setC(219).build()); message.addExtension(repeatedImportMessageExtensionLite, ImportMessageLite.newBuilder().setD(220).build()); + message.addExtension(repeatedLazyMessageExtensionLite, + TestAllTypesLite.NestedMessage.newBuilder().setBb(227).build()); message.addExtension(repeatedNestedEnumExtensionLite, TestAllTypesLite.NestedEnum.BAR); message.addExtension(repeatedForeignEnumExtensionLite, ForeignEnumLite.FOREIGN_LITE_BAR); @@ -2032,6 +2099,8 @@ public final class TestUtil { ForeignMessageLite.newBuilder().setC(319).build()); message.addExtension(repeatedImportMessageExtensionLite, ImportMessageLite.newBuilder().setD(320).build()); + message.addExtension(repeatedLazyMessageExtensionLite, + TestAllTypesLite.NestedMessage.newBuilder().setBb(327).build()); message.addExtension(repeatedNestedEnumExtensionLite, TestAllTypesLite.NestedEnum.BAZ); message.addExtension(repeatedForeignEnumExtensionLite, ForeignEnumLite.FOREIGN_LITE_BAZ); @@ -2098,6 +2167,8 @@ public final class TestUtil { ForeignMessageLite.newBuilder().setC(519).build()); message.setExtension(repeatedImportMessageExtensionLite, 1, ImportMessageLite.newBuilder().setD(520).build()); + message.setExtension(repeatedLazyMessageExtensionLite, 1, + TestAllTypesLite.NestedMessage.newBuilder().setBb(527).build()); message.setExtension(repeatedNestedEnumExtensionLite , 1, TestAllTypesLite.NestedEnum.FOO); message.setExtension(repeatedForeignEnumExtensionLite, 1, ForeignEnumLite.FOREIGN_LITE_FOO); @@ -2168,6 +2239,9 @@ public final class TestUtil { assertEqualsExactType(118, message.getExtension(optionalNestedMessageExtensionLite ).getBb()); assertEqualsExactType(119, message.getExtension(optionalForeignMessageExtensionLite).getC()); assertEqualsExactType(120, message.getExtension(optionalImportMessageExtensionLite ).getD()); + assertEqualsExactType(126, message.getExtension( + optionalPublicImportMessageExtensionLite).getE()); + assertEqualsExactType(127, message.getExtension(optionalLazyMessageExtensionLite).getBb()); assertEqualsExactType(TestAllTypesLite.NestedEnum.BAZ, message.getExtension(optionalNestedEnumExtensionLite)); @@ -2201,6 +2275,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getExtensionCount(repeatedNestedMessageExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignMessageExtensionLite)); Assert.assertEquals(2, message.getExtensionCount(repeatedImportMessageExtensionLite )); + Assert.assertEquals(2, message.getExtensionCount(repeatedLazyMessageExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedNestedEnumExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignEnumExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedImportEnumExtensionLite )); @@ -2228,6 +2303,7 @@ public final class TestUtil { assertEqualsExactType(218, message.getExtension(repeatedNestedMessageExtensionLite ,0).getBb()); assertEqualsExactType(219, message.getExtension(repeatedForeignMessageExtensionLite,0).getC()); assertEqualsExactType(220, message.getExtension(repeatedImportMessageExtensionLite ,0).getD()); + assertEqualsExactType(227, message.getExtension(repeatedLazyMessageExtensionLite ,0).getBb()); assertEqualsExactType(TestAllTypesLite.NestedEnum.BAR, message.getExtension(repeatedNestedEnumExtensionLite, 0)); @@ -2259,6 +2335,7 @@ public final class TestUtil { assertEqualsExactType(318, message.getExtension(repeatedNestedMessageExtensionLite ,1).getBb()); assertEqualsExactType(319, message.getExtension(repeatedForeignMessageExtensionLite,1).getC()); assertEqualsExactType(320, message.getExtension(repeatedImportMessageExtensionLite ,1).getD()); + assertEqualsExactType(327, message.getExtension(repeatedLazyMessageExtensionLite ,1).getBb()); assertEqualsExactType(TestAllTypesLite.NestedEnum.BAZ, message.getExtension(repeatedNestedEnumExtensionLite, 1)); @@ -2348,10 +2425,12 @@ public final class TestUtil { Assert.assertFalse(message.hasExtension(optionalStringExtensionLite )); Assert.assertFalse(message.hasExtension(optionalBytesExtensionLite )); - Assert.assertFalse(message.hasExtension(optionalGroupExtensionLite )); - Assert.assertFalse(message.hasExtension(optionalNestedMessageExtensionLite )); - Assert.assertFalse(message.hasExtension(optionalForeignMessageExtensionLite)); - Assert.assertFalse(message.hasExtension(optionalImportMessageExtensionLite )); + Assert.assertFalse(message.hasExtension(optionalGroupExtensionLite )); + Assert.assertFalse(message.hasExtension(optionalNestedMessageExtensionLite )); + Assert.assertFalse(message.hasExtension(optionalForeignMessageExtensionLite )); + Assert.assertFalse(message.hasExtension(optionalImportMessageExtensionLite )); + Assert.assertFalse(message.hasExtension(optionalPublicImportMessageExtensionLite)); + Assert.assertFalse(message.hasExtension(optionalLazyMessageExtensionLite )); Assert.assertFalse(message.hasExtension(optionalNestedEnumExtensionLite )); Assert.assertFalse(message.hasExtension(optionalForeignEnumExtensionLite)); @@ -2378,15 +2457,20 @@ public final class TestUtil { assertEqualsExactType(ByteString.EMPTY, message.getExtension(optionalBytesExtensionLite)); // Embedded messages should also be clear. - Assert.assertFalse(message.getExtension(optionalGroupExtensionLite ).hasA()); - Assert.assertFalse(message.getExtension(optionalNestedMessageExtensionLite ).hasBb()); - Assert.assertFalse(message.getExtension(optionalForeignMessageExtensionLite).hasC()); - Assert.assertFalse(message.getExtension(optionalImportMessageExtensionLite ).hasD()); + Assert.assertFalse(message.getExtension(optionalGroupExtensionLite ).hasA()); + Assert.assertFalse(message.getExtension(optionalNestedMessageExtensionLite ).hasBb()); + Assert.assertFalse(message.getExtension(optionalForeignMessageExtensionLite ).hasC()); + Assert.assertFalse(message.getExtension(optionalImportMessageExtensionLite ).hasD()); + Assert.assertFalse(message.getExtension(optionalPublicImportMessageExtensionLite).hasE()); + Assert.assertFalse(message.getExtension(optionalLazyMessageExtensionLite ).hasBb()); assertEqualsExactType(0, message.getExtension(optionalGroupExtensionLite ).getA()); assertEqualsExactType(0, message.getExtension(optionalNestedMessageExtensionLite ).getBb()); assertEqualsExactType(0, message.getExtension(optionalForeignMessageExtensionLite).getC()); assertEqualsExactType(0, message.getExtension(optionalImportMessageExtensionLite ).getD()); + assertEqualsExactType(0, message.getExtension( + optionalPublicImportMessageExtensionLite).getE()); + assertEqualsExactType(0, message.getExtension(optionalLazyMessageExtensionLite ).getBb()); // Enums without defaults are set to the first value in the enum. assertEqualsExactType(TestAllTypesLite.NestedEnum.FOO, @@ -2420,6 +2504,7 @@ public final class TestUtil { Assert.assertEquals(0, message.getExtensionCount(repeatedNestedMessageExtensionLite )); Assert.assertEquals(0, message.getExtensionCount(repeatedForeignMessageExtensionLite)); Assert.assertEquals(0, message.getExtensionCount(repeatedImportMessageExtensionLite )); + Assert.assertEquals(0, message.getExtensionCount(repeatedLazyMessageExtensionLite )); Assert.assertEquals(0, message.getExtensionCount(repeatedNestedEnumExtensionLite )); Assert.assertEquals(0, message.getExtensionCount(repeatedForeignEnumExtensionLite )); Assert.assertEquals(0, message.getExtensionCount(repeatedImportEnumExtensionLite )); @@ -2511,6 +2596,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getExtensionCount(repeatedNestedMessageExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignMessageExtensionLite)); Assert.assertEquals(2, message.getExtensionCount(repeatedImportMessageExtensionLite )); + Assert.assertEquals(2, message.getExtensionCount(repeatedLazyMessageExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedNestedEnumExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedForeignEnumExtensionLite )); Assert.assertEquals(2, message.getExtensionCount(repeatedImportEnumExtensionLite )); @@ -2538,6 +2624,7 @@ public final class TestUtil { assertEqualsExactType(218, message.getExtension(repeatedNestedMessageExtensionLite ,0).getBb()); assertEqualsExactType(219, message.getExtension(repeatedForeignMessageExtensionLite,0).getC()); assertEqualsExactType(220, message.getExtension(repeatedImportMessageExtensionLite ,0).getD()); + assertEqualsExactType(227, message.getExtension(repeatedLazyMessageExtensionLite ,0).getBb()); assertEqualsExactType(TestAllTypesLite.NestedEnum.BAR, message.getExtension(repeatedNestedEnumExtensionLite, 0)); @@ -2570,6 +2657,7 @@ public final class TestUtil { assertEqualsExactType(518, message.getExtension(repeatedNestedMessageExtensionLite ,1).getBb()); assertEqualsExactType(519, message.getExtension(repeatedForeignMessageExtensionLite,1).getC()); assertEqualsExactType(520, message.getExtension(repeatedImportMessageExtensionLite ,1).getD()); + assertEqualsExactType(527, message.getExtension(repeatedLazyMessageExtensionLite ,1).getBb()); assertEqualsExactType(TestAllTypesLite.NestedEnum.FOO, message.getExtension(repeatedNestedEnumExtensionLite, 1)); @@ -2674,18 +2762,21 @@ public final class TestUtil { private final Descriptors.FileDescriptor file; private final Descriptors.FileDescriptor importFile; + private final Descriptors.FileDescriptor publicImportFile; private final Descriptors.Descriptor optionalGroup; private final Descriptors.Descriptor repeatedGroup; private final Descriptors.Descriptor nestedMessage; private final Descriptors.Descriptor foreignMessage; private final Descriptors.Descriptor importMessage; + private final Descriptors.Descriptor publicImportMessage; private final Descriptors.FieldDescriptor groupA; private final Descriptors.FieldDescriptor repeatedGroupA; private final Descriptors.FieldDescriptor nestedB; private final Descriptors.FieldDescriptor foreignC; private final Descriptors.FieldDescriptor importD; + private final Descriptors.FieldDescriptor importE; private final Descriptors.EnumDescriptor nestedEnum; private final Descriptors.EnumDescriptor foreignEnum; @@ -2722,6 +2813,7 @@ public final class TestUtil { this.file = baseDescriptor.getFile(); Assert.assertEquals(1, file.getDependencies().size()); this.importFile = file.getDependencies().get(0); + this.publicImportFile = importFile.getDependencies().get(0); Descriptors.Descriptor testAllTypes; if (baseDescriptor.getName() == "TestAllTypes") { @@ -2748,6 +2840,8 @@ public final class TestUtil { this.nestedMessage = testAllTypes.findNestedTypeByName("NestedMessage"); this.foreignMessage = file.findMessageTypeByName("ForeignMessage"); this.importMessage = importFile.findMessageTypeByName("ImportMessage"); + this.publicImportMessage = publicImportFile.findMessageTypeByName( + "PublicImportMessage"); this.nestedEnum = testAllTypes.findEnumTypeByName("NestedEnum"); this.foreignEnum = file.findEnumTypeByName("ForeignEnum"); @@ -2765,6 +2859,7 @@ public final class TestUtil { this.nestedB = nestedMessage .findFieldByName("bb"); this.foreignC = foreignMessage.findFieldByName("c"); this.importD = importMessage .findFieldByName("d"); + this.importE = publicImportMessage.findFieldByName("e"); this.nestedFoo = nestedEnum.findValueByName("FOO"); this.nestedBar = nestedEnum.findValueByName("BAR"); this.nestedBaz = nestedEnum.findValueByName("BAZ"); @@ -2783,6 +2878,7 @@ public final class TestUtil { Assert.assertNotNull(nestedB ); Assert.assertNotNull(foreignC ); Assert.assertNotNull(importD ); + Assert.assertNotNull(importE ); Assert.assertNotNull(nestedFoo ); Assert.assertNotNull(nestedBar ); Assert.assertNotNull(nestedBaz ); @@ -2863,6 +2959,12 @@ public final class TestUtil { message.setField(f("optional_import_message"), newBuilderForField(message, f("optional_import_message")) .setField(importD, 120).build()); + message.setField(f("optional_public_import_message"), + newBuilderForField(message, f("optional_public_import_message")) + .setField(importE, 126).build()); + message.setField(f("optional_lazy_message"), + newBuilderForField(message, f("optional_lazy_message")) + .setField(nestedB, 127).build()); message.setField(f("optional_nested_enum" ), nestedBaz); message.setField(f("optional_foreign_enum"), foreignBaz); @@ -2901,6 +3003,9 @@ public final class TestUtil { message.addRepeatedField(f("repeated_import_message"), newBuilderForField(message, f("repeated_import_message")) .setField(importD, 220).build()); + message.addRepeatedField(f("repeated_lazy_message"), + newBuilderForField(message, f("repeated_lazy_message")) + .setField(nestedB, 227).build()); message.addRepeatedField(f("repeated_nested_enum" ), nestedBar); message.addRepeatedField(f("repeated_foreign_enum"), foreignBar); @@ -2938,6 +3043,9 @@ public final class TestUtil { message.addRepeatedField(f("repeated_import_message"), newBuilderForField(message, f("repeated_import_message")) .setField(importD, 320).build()); + message.addRepeatedField(f("repeated_lazy_message"), + newBuilderForField(message, f("repeated_lazy_message")) + .setField(nestedB, 327).build()); message.addRepeatedField(f("repeated_nested_enum" ), nestedBaz); message.addRepeatedField(f("repeated_foreign_enum"), foreignBaz); @@ -3008,6 +3116,9 @@ public final class TestUtil { message.setRepeatedField(f("repeated_import_message"), 1, newBuilderForField(message, f("repeated_import_message")) .setField(importD, 520).build()); + message.setRepeatedField(f("repeated_lazy_message"), 1, + newBuilderForField(message, f("repeated_lazy_message")) + .setField(nestedB, 527).build()); message.setRepeatedField(f("repeated_nested_enum" ), 1, nestedFoo); message.setRepeatedField(f("repeated_foreign_enum"), 1, foreignFoo); @@ -3092,6 +3203,12 @@ public final class TestUtil { Assert.assertEquals(120, ((Message)message.getField(f("optional_import_message"))) .getField(importD)); + Assert.assertEquals(126, + ((Message)message.getField(f("optional_public_import_message"))) + .getField(importE)); + Assert.assertEquals(127, + ((Message)message.getField(f("optional_lazy_message"))) + .getField(nestedB)); Assert.assertEquals( nestedBaz, message.getField(f("optional_nested_enum" ))); Assert.assertEquals(foreignBaz, message.getField(f("optional_foreign_enum"))); @@ -3122,6 +3239,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_nested_message" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_foreign_message"))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_import_message" ))); + Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_lazy_message" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_nested_enum" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_foreign_enum" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_import_enum" ))); @@ -3157,6 +3275,9 @@ public final class TestUtil { Assert.assertEquals(220, ((Message)message.getRepeatedField(f("repeated_import_message"), 0)) .getField(importD)); + Assert.assertEquals(227, + ((Message)message.getRepeatedField(f("repeated_lazy_message"), 0)) + .getField(nestedB)); Assert.assertEquals( nestedBar, message.getRepeatedField(f("repeated_nested_enum" ),0)); Assert.assertEquals(foreignBar, message.getRepeatedField(f("repeated_foreign_enum"),0)); @@ -3193,6 +3314,9 @@ public final class TestUtil { Assert.assertEquals(320, ((Message)message.getRepeatedField(f("repeated_import_message"), 1)) .getField(importD)); + Assert.assertEquals(327, + ((Message)message.getRepeatedField(f("repeated_lazy_message"), 1)) + .getField(nestedB)); Assert.assertEquals( nestedBaz, message.getRepeatedField(f("repeated_nested_enum" ),1)); Assert.assertEquals(foreignBaz, message.getRepeatedField(f("repeated_foreign_enum"),1)); @@ -3316,6 +3440,12 @@ public final class TestUtil { Assert.assertFalse( ((Message)message.getField(f("optional_import_message"))) .hasField(importD)); + Assert.assertFalse( + ((Message)message.getField(f("optional_public_import_message"))) + .hasField(importE)); + Assert.assertFalse( + ((Message)message.getField(f("optional_lazy_message"))) + .hasField(nestedB)); Assert.assertEquals(0, ((Message)message.getField(f("optionalgroup"))).getField(groupA)); @@ -3328,6 +3458,12 @@ public final class TestUtil { Assert.assertEquals(0, ((Message)message.getField(f("optional_import_message"))) .getField(importD)); + Assert.assertEquals(0, + ((Message)message.getField(f("optional_public_import_message"))) + .getField(importE)); + Assert.assertEquals(0, + ((Message)message.getField(f("optional_lazy_message"))) + .getField(nestedB)); // Enums without defaults are set to the first value in the enum. Assert.assertEquals( nestedFoo, message.getField(f("optional_nested_enum" ))); @@ -3358,6 +3494,7 @@ public final class TestUtil { Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_nested_message" ))); Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_foreign_message"))); Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_import_message" ))); + Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_lazy_message" ))); Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_nested_enum" ))); Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_foreign_enum" ))); Assert.assertEquals(0, message.getRepeatedFieldCount(f("repeated_import_enum" ))); @@ -3442,6 +3579,7 @@ public final class TestUtil { Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_nested_message" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_foreign_message"))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_import_message" ))); + Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_lazy_message" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_nested_enum" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_foreign_enum" ))); Assert.assertEquals(2, message.getRepeatedFieldCount(f("repeated_import_enum" ))); @@ -3477,6 +3615,9 @@ public final class TestUtil { Assert.assertEquals(220, ((Message)message.getRepeatedField(f("repeated_import_message"), 0)) .getField(importD)); + Assert.assertEquals(227, + ((Message)message.getRepeatedField(f("repeated_lazy_message"), 0)) + .getField(nestedB)); Assert.assertEquals( nestedBar, message.getRepeatedField(f("repeated_nested_enum" ),0)); Assert.assertEquals(foreignBar, message.getRepeatedField(f("repeated_foreign_enum"),0)); @@ -3513,6 +3654,9 @@ public final class TestUtil { Assert.assertEquals(520, ((Message)message.getRepeatedField(f("repeated_import_message"), 1)) .getField(importD)); + Assert.assertEquals(527, + ((Message)message.getRepeatedField(f("repeated_lazy_message"), 1)) + .getField(nestedB)); Assert.assertEquals( nestedFoo, message.getRepeatedField(f("repeated_nested_enum" ),1)); Assert.assertEquals(foreignFoo, message.getRepeatedField(f("repeated_foreign_enum"),1)); diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/TextFormatTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/TextFormatTest.java index 47690d1..5323d70 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/TextFormatTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/TextFormatTest.java @@ -121,6 +121,18 @@ public class TextFormatTest extends TestCase { assertEquals(allFieldsSetText, javaText); } + /** Print TestAllTypes as Builder and compare with golden file. */ + public void testPrintMessageBuilder() throws Exception { + String javaText = TextFormat.printToString(TestUtil.getAllSetBuilder()); + + // Java likes to add a trailing ".0" to floats and doubles. C printf + // (with %g format) does not. Our golden files are used for both + // C++ and Java TextFormat classes, so we need to conform. + javaText = javaText.replace(".0\n", "\n"); + + assertEquals(allFieldsSetText, javaText); + } + /** Print TestAllExtensions and compare with golden file. */ public void testPrintExtensions() throws Exception { String javaText = TextFormat.printToString(TestUtil.getAllExtensionsSet()); @@ -749,4 +761,26 @@ public class TextFormatTest extends TestCase { + " 0xabcdef1234567890", TextFormat.shortDebugString(makeUnknownFieldSet())); } + + public void testPrintToUnicodeString() { + assertEquals( + "optional_string: \"abc\u3042efg\"\n" + + "optional_bytes: \"\\343\\201\\202\"\n" + + "repeated_string: \"\u3093XYZ\"\n", + TextFormat.printToUnicodeString(TestAllTypes.newBuilder() + .setOptionalString("abc\u3042efg") + .setOptionalBytes(bytes(0xe3, 0x81, 0x82)) + .addRepeatedString("\u3093XYZ") + .build())); + } + + public void testPrintToUnicodeString_unknown() { + assertEquals( + "1: \"\\343\\201\\202\"\n", + TextFormat.printToUnicodeString(UnknownFieldSet.newBuilder() + .addField(1, + UnknownFieldSet.Field.newBuilder() + .addLengthDelimited(bytes(0xe3, 0x81, 0x82)).build()) + .build())); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/WireFormatTest.java b/third_party/protobuf/java/src/test/java/com/google/protobuf/WireFormatTest.java index 5ea1dd6..94f6213 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/WireFormatTest.java +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/WireFormatTest.java @@ -34,6 +34,7 @@ import junit.framework.TestCase; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; +import java.util.List; import protobuf_unittest.UnittestProto; import protobuf_unittest.UnittestProto.TestAllExtensions; @@ -328,7 +329,17 @@ public class WireFormatTest extends TestCase { private static final int TYPE_ID_2 = TestMessageSetExtension2.getDescriptor().getExtensions().get(0).getNumber(); - public void testSerializeMessageSet() throws Exception { + public void testSerializeMessageSetEagerly() throws Exception { + testSerializeMessageSetWithFlag(true); + } + + public void testSerializeMessageSetNotEagerly() throws Exception { + testSerializeMessageSetWithFlag(false); + } + + private void testSerializeMessageSetWithFlag(boolean eagerParsing) + throws Exception { + ExtensionRegistryLite.setEagerlyParseMessageSets(eagerParsing); // Set up a TestMessageSet with two known messages and an unknown one. TestMessageSet messageSet = TestMessageSet.newBuilder() @@ -372,7 +383,17 @@ public class WireFormatTest extends TestCase { assertEquals("bar", raw.getItem(2).getMessage().toStringUtf8()); } - public void testParseMessageSet() throws Exception { + public void testParseMessageSetEagerly() throws Exception { + testParseMessageSetWithFlag(true); + } + + public void testParseMessageSetNotEagerly()throws Exception { + testParseMessageSetWithFlag(false); + } + + private void testParseMessageSetWithFlag(boolean eagerParsing) + throws Exception { + ExtensionRegistryLite.setEagerlyParseMessageSets(eagerParsing); ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance(); extensionRegistry.add(TestMessageSetExtension1.messageSetExtension); extensionRegistry.add(TestMessageSetExtension2.messageSetExtension); @@ -424,4 +445,136 @@ public class WireFormatTest extends TestCase { assertEquals(1, field.getLengthDelimitedList().size()); assertEquals("bar", field.getLengthDelimitedList().get(0).toStringUtf8()); } + + public void testParseMessageSetExtensionEagerly() throws Exception { + testParseMessageSetExtensionWithFlag(true); + } + + public void testParseMessageSetExtensionNotEagerly() throws Exception { + testParseMessageSetExtensionWithFlag(false); + } + + private void testParseMessageSetExtensionWithFlag(boolean eagerParsing) + throws Exception { + ExtensionRegistryLite.setEagerlyParseMessageSets(eagerParsing); + ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance(); + extensionRegistry.add(TestMessageSetExtension1.messageSetExtension); + + // Set up a RawMessageSet with a known messages. + int TYPE_ID_1 = + TestMessageSetExtension1 + .getDescriptor().getExtensions().get(0).getNumber(); + RawMessageSet raw = + RawMessageSet.newBuilder() + .addItem( + RawMessageSet.Item.newBuilder() + .setTypeId(TYPE_ID_1) + .setMessage( + TestMessageSetExtension1.newBuilder() + .setI(123) + .build().toByteString()) + .build()) + .build(); + + ByteString data = raw.toByteString(); + + // Parse as a TestMessageSet and check the contents. + TestMessageSet messageSet = + TestMessageSet.parseFrom(data, extensionRegistry); + assertEquals(123, messageSet.getExtension( + TestMessageSetExtension1.messageSetExtension).getI()); + } + + public void testMergeLazyMessageSetExtensionEagerly() throws Exception { + testMergeLazyMessageSetExtensionWithFlag(true); + } + + public void testMergeLazyMessageSetExtensionNotEagerly() throws Exception { + testMergeLazyMessageSetExtensionWithFlag(false); + } + + private void testMergeLazyMessageSetExtensionWithFlag(boolean eagerParsing) + throws Exception { + ExtensionRegistryLite.setEagerlyParseMessageSets(eagerParsing); + ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance(); + extensionRegistry.add(TestMessageSetExtension1.messageSetExtension); + + // Set up a RawMessageSet with a known messages. + int TYPE_ID_1 = + TestMessageSetExtension1 + .getDescriptor().getExtensions().get(0).getNumber(); + RawMessageSet raw = + RawMessageSet.newBuilder() + .addItem( + RawMessageSet.Item.newBuilder() + .setTypeId(TYPE_ID_1) + .setMessage( + TestMessageSetExtension1.newBuilder() + .setI(123) + .build().toByteString()) + .build()) + .build(); + + ByteString data = raw.toByteString(); + + // Parse as a TestMessageSet and store value into lazy field + TestMessageSet messageSet = + TestMessageSet.parseFrom(data, extensionRegistry); + // Merge lazy field check the contents. + messageSet = + messageSet.toBuilder().mergeFrom(data, extensionRegistry).build(); + assertEquals(123, messageSet.getExtension( + TestMessageSetExtension1.messageSetExtension).getI()); + } + + public void testMergeMessageSetExtensionEagerly() throws Exception { + testMergeMessageSetExtensionWithFlag(true); + } + + public void testMergeMessageSetExtensionNotEagerly() throws Exception { + testMergeMessageSetExtensionWithFlag(false); + } + + private void testMergeMessageSetExtensionWithFlag(boolean eagerParsing) + throws Exception { + ExtensionRegistryLite.setEagerlyParseMessageSets(eagerParsing); + ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance(); + extensionRegistry.add(TestMessageSetExtension1.messageSetExtension); + + // Set up a RawMessageSet with a known messages. + int TYPE_ID_1 = + TestMessageSetExtension1 + .getDescriptor().getExtensions().get(0).getNumber(); + RawMessageSet raw = + RawMessageSet.newBuilder() + .addItem( + RawMessageSet.Item.newBuilder() + .setTypeId(TYPE_ID_1) + .setMessage( + TestMessageSetExtension1.newBuilder() + .setI(123) + .build().toByteString()) + .build()) + .build(); + + // Serialize RawMessageSet unnormally (message value before type id) + ByteString.CodedBuilder out = ByteString.newCodedBuilder( + raw.getSerializedSize()); + CodedOutputStream output = out.getCodedOutput(); + List<RawMessageSet.Item> items = raw.getItemList(); + for (int i = 0; i < items.size(); i++) { + RawMessageSet.Item item = items.get(i); + output.writeTag(1, WireFormat.WIRETYPE_START_GROUP); + output.writeBytes(3, item.getMessage()); + output.writeInt32(2, item.getTypeId()); + output.writeTag(1, WireFormat.WIRETYPE_END_GROUP); + } + ByteString data = out.build(); + + // Merge bytes into TestMessageSet and check the contents. + TestMessageSet messageSet = + TestMessageSet.newBuilder().mergeFrom(data, extensionRegistry).build(); + assertEquals(123, messageSet.getExtension( + TestMessageSetExtension1.messageSetExtension).getI()); + } } diff --git a/third_party/protobuf/java/src/test/java/com/google/protobuf/test_bad_identifiers.proto b/third_party/protobuf/java/src/test/java/com/google/protobuf/test_bad_identifiers.proto index 9610e9d..6e67d97 100644 --- a/third_party/protobuf/java/src/test/java/com/google/protobuf/test_bad_identifiers.proto +++ b/third_party/protobuf/java/src/test/java/com/google/protobuf/test_bad_identifiers.proto @@ -46,6 +46,23 @@ option java_outer_classname = "TestBadIdentifiersProto"; message TestMessage { } +message Descriptor { + option no_standard_descriptor_accessor = true; + optional string descriptor = 1; + message NestedDescriptor { + option no_standard_descriptor_accessor = true; + optional string descriptor = 1; + } + optional NestedDescriptor nested_descriptor = 2; +} + +message Parser { + enum ParserEnum { + PARSER = 1; + } + optional ParserEnum parser = 1; +} + message Deprecated { enum TestEnum { FOO = 1; @@ -62,6 +79,7 @@ message Override { message Object { optional int32 object = 1; + optional string string_object = 2; } message String { diff --git a/third_party/protobuf/protobuf.gyp b/third_party/protobuf/protobuf.gyp index df0f6e5..44bcdd6 100644 --- a/third_party/protobuf/protobuf.gyp +++ b/third_party/protobuf/protobuf.gyp @@ -22,6 +22,7 @@ 4244, # implicit conversion, possible loss of data 4355, # 'this' used in base member initializer list 4267, # size_t to int truncation + 4291, # no matching operator delete for a placement new ], 'defines!': [ 'WIN32_LEAN_AND_MEAN', # Protobuf defines this itself. @@ -161,6 +162,7 @@ 'src/google/protobuf/descriptor.pb.h', 'src/google/protobuf/descriptor_database.h', 'src/google/protobuf/dynamic_message.h', + 'src/google/protobuf/generated_enum_reflection.h', 'src/google/protobuf/generated_message_reflection.h', 'src/google/protobuf/message.h', 'src/google/protobuf/reflection_ops.h', @@ -174,17 +176,27 @@ 'src/google/protobuf/compiler/code_generator.h', 'src/google/protobuf/compiler/command_line_interface.h', 'src/google/protobuf/compiler/importer.h', + 'src/google/protobuf/compiler/java/java_doc_comment.cc', + 'src/google/protobuf/compiler/java/java_doc_comment.h', 'src/google/protobuf/compiler/parser.h', 'src/google/protobuf/stubs/strutil.cc', 'src/google/protobuf/stubs/strutil.h', 'src/google/protobuf/stubs/substitute.cc', 'src/google/protobuf/stubs/substitute.h', + 'src/google/protobuf/stubs/stl_util.h', + 'src/google/protobuf/stubs/stringprintf.cc', + 'src/google/protobuf/stubs/stringprintf.h', 'src/google/protobuf/stubs/structurally_valid.cc', + 'src/google/protobuf/stubs/template_util.h', + 'src/google/protobuf/stubs/type_traits.h', + 'src/google/protobuf/descriptor.cc', 'src/google/protobuf/descriptor.pb.cc', 'src/google/protobuf/descriptor_database.cc', 'src/google/protobuf/dynamic_message.cc', + 'src/google/protobuf/extension_set.cc', + 'src/google/protobuf/extension_set.h', 'src/google/protobuf/extension_set_heavy.cc', 'src/google/protobuf/generated_message_reflection.cc', 'src/google/protobuf/message.cc', @@ -314,7 +326,10 @@ 'files': [ 'python/google/protobuf/__init__.py', 'python/google/protobuf/descriptor.py', + 'python/google/protobuf/descriptor_database.py', + 'python/google/protobuf/descriptor_pool.py', 'python/google/protobuf/message.py', + 'python/google/protobuf/message_factory.py', 'python/google/protobuf/reflection.py', 'python/google/protobuf/service.py', 'python/google/protobuf/service_reflection.py', @@ -340,6 +355,7 @@ 'python/google/protobuf/internal/cpp_message.py', 'python/google/protobuf/internal/decoder.py', 'python/google/protobuf/internal/encoder.py', + 'python/google/protobuf/internal/enum_type_wrapper.py', 'python/google/protobuf/internal/generator_test.py', 'python/google/protobuf/internal/message_listener.py', 'python/google/protobuf/internal/python_message.py', diff --git a/third_party/protobuf/python/google/protobuf/descriptor.py b/third_party/protobuf/python/google/protobuf/descriptor.py index cf609be..eb13eda 100755 --- a/third_party/protobuf/python/google/protobuf/descriptor.py +++ b/third_party/protobuf/python/google/protobuf/descriptor.py @@ -39,13 +39,20 @@ from google.protobuf.internal import api_implementation if api_implementation.Type() == 'cpp': - from google.protobuf.internal import cpp_message + if api_implementation.Version() == 2: + from google.protobuf.internal.cpp import _message + else: + from google.protobuf.internal import cpp_message class Error(Exception): """Base error for this module.""" +class TypeTransformationError(Error): + """Error transforming between python proto type and corresponding C++ type.""" + + class DescriptorBase(object): """Descriptors base class. @@ -72,6 +79,18 @@ class DescriptorBase(object): # Does this descriptor have non-default options? self.has_options = options is not None + def _SetOptions(self, options, options_class_name): + """Sets the descriptor's options + + This function is used in generated proto2 files to update descriptor + options. It must not be used outside proto2. + """ + self._options = options + self._options_class_name = options_class_name + + # Does this descriptor have non-default options? + self.has_options = options is not None + def GetOptions(self): """Retrieves descriptor options. @@ -250,6 +269,24 @@ class Descriptor(_NestedDescriptorBase): self._serialized_start = serialized_start self._serialized_end = serialized_end + def EnumValueName(self, enum, value): + """Returns the string name of an enum value. + + This is just a small helper method to simplify a common operation. + + Args: + enum: string name of the Enum. + value: int, value of the enum. + + Returns: + string name of the enum value. + + Raises: + KeyError if either the Enum doesn't exist or the value is not a valid + value for the enum. + """ + return self.enum_types_by_name[enum].values_by_number[value].name + def CopyToProto(self, proto): """Copies this to a descriptor_pb2.DescriptorProto. @@ -275,7 +312,7 @@ class FieldDescriptor(DescriptorBase): """Descriptor for a single field in a .proto file. - A FieldDescriptor instance has the following attriubtes: + A FieldDescriptor instance has the following attributes: name: (str) Name of this field, exactly as it appears in .proto. full_name: (str) Name of this field, including containing scope. This is @@ -358,6 +395,27 @@ class FieldDescriptor(DescriptorBase): CPPTYPE_MESSAGE = 10 MAX_CPPTYPE = 10 + _PYTHON_TO_CPP_PROTO_TYPE_MAP = { + TYPE_DOUBLE: CPPTYPE_DOUBLE, + TYPE_FLOAT: CPPTYPE_FLOAT, + TYPE_ENUM: CPPTYPE_ENUM, + TYPE_INT64: CPPTYPE_INT64, + TYPE_SINT64: CPPTYPE_INT64, + TYPE_SFIXED64: CPPTYPE_INT64, + TYPE_UINT64: CPPTYPE_UINT64, + TYPE_FIXED64: CPPTYPE_UINT64, + TYPE_INT32: CPPTYPE_INT32, + TYPE_SFIXED32: CPPTYPE_INT32, + TYPE_SINT32: CPPTYPE_INT32, + TYPE_UINT32: CPPTYPE_UINT32, + TYPE_FIXED32: CPPTYPE_UINT32, + TYPE_BYTES: CPPTYPE_STRING, + TYPE_STRING: CPPTYPE_STRING, + TYPE_BOOL: CPPTYPE_BOOL, + TYPE_MESSAGE: CPPTYPE_MESSAGE, + TYPE_GROUP: CPPTYPE_MESSAGE + } + # Must be consistent with C++ FieldDescriptor::Label enum in # descriptor.h. # @@ -395,12 +453,38 @@ class FieldDescriptor(DescriptorBase): self.extension_scope = extension_scope if api_implementation.Type() == 'cpp': if is_extension: - self._cdescriptor = cpp_message.GetExtensionDescriptor(full_name) + if api_implementation.Version() == 2: + self._cdescriptor = _message.GetExtensionDescriptor(full_name) + else: + self._cdescriptor = cpp_message.GetExtensionDescriptor(full_name) else: - self._cdescriptor = cpp_message.GetFieldDescriptor(full_name) + if api_implementation.Version() == 2: + self._cdescriptor = _message.GetFieldDescriptor(full_name) + else: + self._cdescriptor = cpp_message.GetFieldDescriptor(full_name) else: self._cdescriptor = None + @staticmethod + def ProtoTypeToCppProtoType(proto_type): + """Converts from a Python proto type to a C++ Proto Type. + + The Python ProtocolBuffer classes specify both the 'Python' datatype and the + 'C++' datatype - and they're not the same. This helper method should + translate from one to another. + + Args: + proto_type: the Python proto type (descriptor.FieldDescriptor.TYPE_*) + Returns: + descriptor.FieldDescriptor.CPPTYPE_*, the C++ type. + Raises: + TypeTransformationError: when the Python proto type isn't known. + """ + try: + return FieldDescriptor._PYTHON_TO_CPP_PROTO_TYPE_MAP[proto_type] + except KeyError: + raise TypeTransformationError('Unknown proto_type: %s' % proto_type) + class EnumDescriptor(_NestedDescriptorBase): @@ -577,7 +661,10 @@ class FileDescriptor(DescriptorBase): self.serialized_pb = serialized_pb if (api_implementation.Type() == 'cpp' and self.serialized_pb is not None): - cpp_message.BuildFile(self.serialized_pb) + if api_implementation.Version() == 2: + _message.BuildFile(self.serialized_pb) + else: + cpp_message.BuildFile(self.serialized_pb) def CopyToProto(self, proto): """Copies this to a descriptor_pb2.FileDescriptorProto. @@ -596,3 +683,31 @@ def _ParseOptions(message, string): """ message.ParseFromString(string) return message + + +def MakeDescriptor(desc_proto, package=''): + """Make a protobuf Descriptor given a DescriptorProto protobuf. + + Args: + desc_proto: The descriptor_pb2.DescriptorProto protobuf message. + package: Optional package name for the new message Descriptor (string). + + Returns: + A Descriptor for protobuf messages. + """ + full_message_name = [desc_proto.name] + if package: full_message_name.insert(0, package) + fields = [] + for field_proto in desc_proto.field: + full_name = '.'.join(full_message_name + [field_proto.name]) + field = FieldDescriptor( + field_proto.name, full_name, field_proto.number - 1, + field_proto.number, field_proto.type, + FieldDescriptor.ProtoTypeToCppProtoType(field_proto.type), + field_proto.label, None, None, None, None, False, None, + has_default_value=False) + fields.append(field) + + desc_name = '.'.join(full_message_name) + return Descriptor(desc_proto.name, desc_name, None, None, fields, + [], [], []) diff --git a/third_party/protobuf/python/google/protobuf/descriptor_database.py b/third_party/protobuf/python/google/protobuf/descriptor_database.py new file mode 100644 index 0000000..8665d3c --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/descriptor_database.py @@ -0,0 +1,120 @@ +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Provides a container for DescriptorProtos.""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + + +class DescriptorDatabase(object): + """A container accepting FileDescriptorProtos and maps DescriptorProtos.""" + + def __init__(self): + self._file_desc_protos_by_file = {} + self._file_desc_protos_by_symbol = {} + + def Add(self, file_desc_proto): + """Adds the FileDescriptorProto and its types to this database. + + Args: + file_desc_proto: The FileDescriptorProto to add. + """ + + self._file_desc_protos_by_file[file_desc_proto.name] = file_desc_proto + package = file_desc_proto.package + for message in file_desc_proto.message_type: + self._file_desc_protos_by_symbol.update( + (name, file_desc_proto) for name in _ExtractSymbols(message, package)) + for enum in file_desc_proto.enum_type: + self._file_desc_protos_by_symbol[ + '.'.join((package, enum.name))] = file_desc_proto + + def FindFileByName(self, name): + """Finds the file descriptor proto by file name. + + Typically the file name is a relative path ending to a .proto file. The + proto with the given name will have to have been added to this database + using the Add method or else an error will be raised. + + Args: + name: The file name to find. + + Returns: + The file descriptor proto matching the name. + + Raises: + KeyError if no file by the given name was added. + """ + + return self._file_desc_protos_by_file[name] + + def FindFileContainingSymbol(self, symbol): + """Finds the file descriptor proto containing the specified symbol. + + The symbol should be a fully qualified name including the file descriptor's + package and any containing messages. Some examples: + + 'some.package.name.Message' + 'some.package.name.Message.NestedEnum' + + The file descriptor proto containing the specified symbol must be added to + this database using the Add method or else an error will be raised. + + Args: + symbol: The fully qualified symbol name. + + Returns: + The file descriptor proto containing the symbol. + + Raises: + KeyError if no file contains the specified symbol. + """ + + return self._file_desc_protos_by_symbol[symbol] + + +def _ExtractSymbols(desc_proto, package): + """Pulls out all the symbols from a descriptor proto. + + Args: + desc_proto: The proto to extract symbols from. + package: The package containing the descriptor type. + + Yields: + The fully qualified name found in the descriptor. + """ + + message_name = '.'.join((package, desc_proto.name)) + yield message_name + for nested_type in desc_proto.nested_type: + for symbol in _ExtractSymbols(nested_type, message_name): + yield symbol + for enum_type in desc_proto.enum_type: + yield '.'.join((message_name, enum_type.name)) diff --git a/third_party/protobuf/python/google/protobuf/descriptor_pool.py b/third_party/protobuf/python/google/protobuf/descriptor_pool.py new file mode 100644 index 0000000..8f1f445 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/descriptor_pool.py @@ -0,0 +1,527 @@ +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Provides DescriptorPool to use as a container for proto2 descriptors. + +The DescriptorPool is used in conjection with a DescriptorDatabase to maintain +a collection of protocol buffer descriptors for use when dynamically creating +message types at runtime. + +For most applications protocol buffers should be used via modules generated by +the protocol buffer compiler tool. This should only be used when the type of +protocol buffers used in an application or library cannot be predetermined. + +Below is a straightforward example on how to use this class: + + pool = DescriptorPool() + file_descriptor_protos = [ ... ] + for file_descriptor_proto in file_descriptor_protos: + pool.Add(file_descriptor_proto) + my_message_descriptor = pool.FindMessageTypeByName('some.package.MessageType') + +The message descriptor can be used in conjunction with the message_factory +module in order to create a protocol buffer class that can be encoded and +decoded. +""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + +from google.protobuf import descriptor_pb2 +from google.protobuf import descriptor +from google.protobuf import descriptor_database + + +class DescriptorPool(object): + """A collection of protobufs dynamically constructed by descriptor protos.""" + + def __init__(self, descriptor_db=None): + """Initializes a Pool of proto buffs. + + The descriptor_db argument to the constructor is provided to allow + specialized file descriptor proto lookup code to be triggered on demand. An + example would be an implementation which will read and compile a file + specified in a call to FindFileByName() and not require the call to Add() + at all. Results from this database will be cached internally here as well. + + Args: + descriptor_db: A secondary source of file descriptors. + """ + + self._internal_db = descriptor_database.DescriptorDatabase() + self._descriptor_db = descriptor_db + self._descriptors = {} + self._enum_descriptors = {} + self._file_descriptors = {} + + def Add(self, file_desc_proto): + """Adds the FileDescriptorProto and its types to this pool. + + Args: + file_desc_proto: The FileDescriptorProto to add. + """ + + self._internal_db.Add(file_desc_proto) + + def FindFileByName(self, file_name): + """Gets a FileDescriptor by file name. + + Args: + file_name: The path to the file to get a descriptor for. + + Returns: + A FileDescriptor for the named file. + + Raises: + KeyError: if the file can not be found in the pool. + """ + + try: + file_proto = self._internal_db.FindFileByName(file_name) + except KeyError as error: + if self._descriptor_db: + file_proto = self._descriptor_db.FindFileByName(file_name) + else: + raise error + if not file_proto: + raise KeyError('Cannot find a file named %s' % file_name) + return self._ConvertFileProtoToFileDescriptor(file_proto) + + def FindFileContainingSymbol(self, symbol): + """Gets the FileDescriptor for the file containing the specified symbol. + + Args: + symbol: The name of the symbol to search for. + + Returns: + A FileDescriptor that contains the specified symbol. + + Raises: + KeyError: if the file can not be found in the pool. + """ + + try: + file_proto = self._internal_db.FindFileContainingSymbol(symbol) + except KeyError as error: + if self._descriptor_db: + file_proto = self._descriptor_db.FindFileContainingSymbol(symbol) + else: + raise error + if not file_proto: + raise KeyError('Cannot find a file containing %s' % symbol) + return self._ConvertFileProtoToFileDescriptor(file_proto) + + def FindMessageTypeByName(self, full_name): + """Loads the named descriptor from the pool. + + Args: + full_name: The full name of the descriptor to load. + + Returns: + The descriptor for the named type. + """ + + full_name = full_name.lstrip('.') # fix inconsistent qualified name formats + if full_name not in self._descriptors: + self.FindFileContainingSymbol(full_name) + return self._descriptors[full_name] + + def FindEnumTypeByName(self, full_name): + """Loads the named enum descriptor from the pool. + + Args: + full_name: The full name of the enum descriptor to load. + + Returns: + The enum descriptor for the named type. + """ + + full_name = full_name.lstrip('.') # fix inconsistent qualified name formats + if full_name not in self._enum_descriptors: + self.FindFileContainingSymbol(full_name) + return self._enum_descriptors[full_name] + + def _ConvertFileProtoToFileDescriptor(self, file_proto): + """Creates a FileDescriptor from a proto or returns a cached copy. + + This method also has the side effect of loading all the symbols found in + the file into the appropriate dictionaries in the pool. + + Args: + file_proto: The proto to convert. + + Returns: + A FileDescriptor matching the passed in proto. + """ + + if file_proto.name not in self._file_descriptors: + file_descriptor = descriptor.FileDescriptor( + name=file_proto.name, + package=file_proto.package, + options=file_proto.options, + serialized_pb=file_proto.SerializeToString()) + scope = {} + dependencies = list(self._GetDeps(file_proto)) + + for dependency in dependencies: + dep_desc = self.FindFileByName(dependency.name) + dep_proto = descriptor_pb2.FileDescriptorProto.FromString( + dep_desc.serialized_pb) + package = '.' + dep_proto.package + package_prefix = package + '.' + + def _strip_package(symbol): + if symbol.startswith(package_prefix): + return symbol[len(package_prefix):] + return symbol + + symbols = list(self._ExtractSymbols(dep_proto.message_type, package)) + scope.update(symbols) + scope.update((_strip_package(k), v) for k, v in symbols) + + symbols = list(self._ExtractEnums(dep_proto.enum_type, package)) + scope.update(symbols) + scope.update((_strip_package(k), v) for k, v in symbols) + + for message_type in file_proto.message_type: + message_desc = self._ConvertMessageDescriptor( + message_type, file_proto.package, file_descriptor, scope) + file_descriptor.message_types_by_name[message_desc.name] = message_desc + for enum_type in file_proto.enum_type: + self._ConvertEnumDescriptor(enum_type, file_proto.package, + file_descriptor, None, scope) + for desc_proto in self._ExtractMessages(file_proto.message_type): + self._SetFieldTypes(desc_proto, scope) + + for desc_proto in file_proto.message_type: + desc = scope[desc_proto.name] + file_descriptor.message_types_by_name[desc_proto.name] = desc + self.Add(file_proto) + self._file_descriptors[file_proto.name] = file_descriptor + + return self._file_descriptors[file_proto.name] + + def _ConvertMessageDescriptor(self, desc_proto, package=None, file_desc=None, + scope=None): + """Adds the proto to the pool in the specified package. + + Args: + desc_proto: The descriptor_pb2.DescriptorProto protobuf message. + package: The package the proto should be located in. + file_desc: The file containing this message. + scope: Dict mapping short and full symbols to message and enum types. + + Returns: + The added descriptor. + """ + + if package: + desc_name = '.'.join((package, desc_proto.name)) + else: + desc_name = desc_proto.name + + if file_desc is None: + file_name = None + else: + file_name = file_desc.name + + if scope is None: + scope = {} + + nested = [ + self._ConvertMessageDescriptor(nested, desc_name, file_desc, scope) + for nested in desc_proto.nested_type] + enums = [ + self._ConvertEnumDescriptor(enum, desc_name, file_desc, None, scope) + for enum in desc_proto.enum_type] + fields = [self._MakeFieldDescriptor(field, desc_name, index) + for index, field in enumerate(desc_proto.field)] + extensions = [self._MakeFieldDescriptor(extension, desc_name, True) + for index, extension in enumerate(desc_proto.extension)] + extension_ranges = [(r.start, r.end) for r in desc_proto.extension_range] + if extension_ranges: + is_extendable = True + else: + is_extendable = False + desc = descriptor.Descriptor( + name=desc_proto.name, + full_name=desc_name, + filename=file_name, + containing_type=None, + fields=fields, + nested_types=nested, + enum_types=enums, + extensions=extensions, + options=desc_proto.options, + is_extendable=is_extendable, + extension_ranges=extension_ranges, + file=file_desc, + serialized_start=None, + serialized_end=None) + for nested in desc.nested_types: + nested.containing_type = desc + for enum in desc.enum_types: + enum.containing_type = desc + scope[desc_proto.name] = desc + scope['.' + desc_name] = desc + self._descriptors[desc_name] = desc + return desc + + def _ConvertEnumDescriptor(self, enum_proto, package=None, file_desc=None, + containing_type=None, scope=None): + """Make a protobuf EnumDescriptor given an EnumDescriptorProto protobuf. + + Args: + enum_proto: The descriptor_pb2.EnumDescriptorProto protobuf message. + package: Optional package name for the new message EnumDescriptor. + file_desc: The file containing the enum descriptor. + containing_type: The type containing this enum. + scope: Scope containing available types. + + Returns: + The added descriptor + """ + + if package: + enum_name = '.'.join((package, enum_proto.name)) + else: + enum_name = enum_proto.name + + if file_desc is None: + file_name = None + else: + file_name = file_desc.name + + values = [self._MakeEnumValueDescriptor(value, index) + for index, value in enumerate(enum_proto.value)] + desc = descriptor.EnumDescriptor(name=enum_proto.name, + full_name=enum_name, + filename=file_name, + file=file_desc, + values=values, + containing_type=containing_type, + options=enum_proto.options) + scope[enum_proto.name] = desc + scope['.%s' % enum_name] = desc + self._enum_descriptors[enum_name] = desc + return desc + + def _MakeFieldDescriptor(self, field_proto, message_name, index, + is_extension=False): + """Creates a field descriptor from a FieldDescriptorProto. + + For message and enum type fields, this method will do a look up + in the pool for the appropriate descriptor for that type. If it + is unavailable, it will fall back to the _source function to + create it. If this type is still unavailable, construction will + fail. + + Args: + field_proto: The proto describing the field. + message_name: The name of the containing message. + index: Index of the field + is_extension: Indication that this field is for an extension. + + Returns: + An initialized FieldDescriptor object + """ + + if message_name: + full_name = '.'.join((message_name, field_proto.name)) + else: + full_name = field_proto.name + + return descriptor.FieldDescriptor( + name=field_proto.name, + full_name=full_name, + index=index, + number=field_proto.number, + type=field_proto.type, + cpp_type=None, + message_type=None, + enum_type=None, + containing_type=None, + label=field_proto.label, + has_default_value=False, + default_value=None, + is_extension=is_extension, + extension_scope=None, + options=field_proto.options) + + def _SetFieldTypes(self, desc_proto, scope): + """Sets the field's type, cpp_type, message_type and enum_type. + + Args: + desc_proto: The message descriptor to update. + scope: Enclosing scope of available types. + """ + + desc = scope[desc_proto.name] + for field_proto, field_desc in zip(desc_proto.field, desc.fields): + if field_proto.type_name: + type_name = field_proto.type_name + if type_name not in scope: + type_name = '.' + type_name + desc = scope[type_name] + else: + desc = None + + if not field_proto.HasField('type'): + if isinstance(desc, descriptor.Descriptor): + field_proto.type = descriptor.FieldDescriptor.TYPE_MESSAGE + else: + field_proto.type = descriptor.FieldDescriptor.TYPE_ENUM + + field_desc.cpp_type = descriptor.FieldDescriptor.ProtoTypeToCppProtoType( + field_proto.type) + + if (field_proto.type == descriptor.FieldDescriptor.TYPE_MESSAGE + or field_proto.type == descriptor.FieldDescriptor.TYPE_GROUP): + field_desc.message_type = desc + + if field_proto.type == descriptor.FieldDescriptor.TYPE_ENUM: + field_desc.enum_type = desc + + if field_proto.label == descriptor.FieldDescriptor.LABEL_REPEATED: + field_desc.has_default = False + field_desc.default_value = [] + elif field_proto.HasField('default_value'): + field_desc.has_default = True + if (field_proto.type == descriptor.FieldDescriptor.TYPE_DOUBLE or + field_proto.type == descriptor.FieldDescriptor.TYPE_FLOAT): + field_desc.default_value = float(field_proto.default_value) + elif field_proto.type == descriptor.FieldDescriptor.TYPE_STRING: + field_desc.default_value = field_proto.default_value + elif field_proto.type == descriptor.FieldDescriptor.TYPE_BOOL: + field_desc.default_value = field_proto.default_value.lower() == 'true' + elif field_proto.type == descriptor.FieldDescriptor.TYPE_ENUM: + field_desc.default_value = field_desc.enum_type.values_by_name[ + field_proto.default_value].index + else: + field_desc.default_value = int(field_proto.default_value) + else: + field_desc.has_default = False + field_desc.default_value = None + + field_desc.type = field_proto.type + + for nested_type in desc_proto.nested_type: + self._SetFieldTypes(nested_type, scope) + + def _MakeEnumValueDescriptor(self, value_proto, index): + """Creates a enum value descriptor object from a enum value proto. + + Args: + value_proto: The proto describing the enum value. + index: The index of the enum value. + + Returns: + An initialized EnumValueDescriptor object. + """ + + return descriptor.EnumValueDescriptor( + name=value_proto.name, + index=index, + number=value_proto.number, + options=value_proto.options, + type=None) + + def _ExtractSymbols(self, desc_protos, package): + """Pulls out all the symbols from descriptor protos. + + Args: + desc_protos: The protos to extract symbols from. + package: The package containing the descriptor type. + Yields: + A two element tuple of the type name and descriptor object. + """ + + for desc_proto in desc_protos: + if package: + message_name = '.'.join((package, desc_proto.name)) + else: + message_name = desc_proto.name + message_desc = self.FindMessageTypeByName(message_name) + yield (message_name, message_desc) + for symbol in self._ExtractSymbols(desc_proto.nested_type, message_name): + yield symbol + for symbol in self._ExtractEnums(desc_proto.enum_type, message_name): + yield symbol + + def _ExtractEnums(self, enum_protos, package): + """Pulls out all the symbols from enum protos. + + Args: + enum_protos: The protos to extract symbols from. + package: The package containing the enum type. + + Yields: + A two element tuple of the type name and enum descriptor object. + """ + + for enum_proto in enum_protos: + if package: + enum_name = '.'.join((package, enum_proto.name)) + else: + enum_name = enum_proto.name + enum_desc = self.FindEnumTypeByName(enum_name) + yield (enum_name, enum_desc) + + def _ExtractMessages(self, desc_protos): + """Pulls out all the message protos from descriptos. + + Args: + desc_protos: The protos to extract symbols from. + + Yields: + Descriptor protos. + """ + + for desc_proto in desc_protos: + yield desc_proto + for message in self._ExtractMessages(desc_proto.nested_type): + yield message + + def _GetDeps(self, file_proto): + """Recursively finds dependencies for file protos. + + Args: + file_proto: The proto to get dependencies from. + + Yields: + Each direct and indirect dependency. + """ + + for dependency in file_proto.dependency: + dep_desc = self.FindFileByName(dependency) + dep_proto = descriptor_pb2.FileDescriptorProto.FromString( + dep_desc.serialized_pb) + yield dep_proto + for parent_dep in self._GetDeps(dep_proto): + yield parent_dep diff --git a/third_party/protobuf/python/google/protobuf/internal/api_implementation.py b/third_party/protobuf/python/google/protobuf/internal/api_implementation.py index b3e412e..ce02a32 100644 --- a/third_party/protobuf/python/google/protobuf/internal/api_implementation.py +++ b/third_party/protobuf/python/google/protobuf/internal/api_implementation.py @@ -56,9 +56,32 @@ if _implementation_type != 'python': # _implementation_type = 'python' +# This environment variable can be used to switch between the two +# 'cpp' implementations. Right now only 1 and 2 are valid values. Any +# other value will be ignored. +_implementation_version_str = os.getenv( + 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION', + '1') + + +if _implementation_version_str not in ('1', '2'): + raise ValueError( + "unsupported PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION: '" + + _implementation_version_str + "' (supported versions: 1, 2)" + ) + + +_implementation_version = int(_implementation_version_str) + + + # Usage of this function is discouraged. Clients shouldn't care which # implementation of the API is in use. Note that there is no guarantee # that differences between APIs will be maintained. # Please don't use this function if possible. def Type(): return _implementation_type + +# See comment on 'Type' above. +def Version(): + return _implementation_version diff --git a/third_party/protobuf/python/google/protobuf/internal/containers.py b/third_party/protobuf/python/google/protobuf/internal/containers.py index 097a3c2..34b35f8 100755 --- a/third_party/protobuf/python/google/protobuf/internal/containers.py +++ b/third_party/protobuf/python/google/protobuf/internal/containers.py @@ -78,8 +78,13 @@ class BaseContainer(object): def __repr__(self): return repr(self._values) - def sort(self, sort_function=cmp): - self._values.sort(sort_function) + def sort(self, *args, **kwargs): + # Continue to support the old sort_function keyword argument. + # This is expected to be a rare occurrence, so use LBYL to avoid + # the overhead of actually catching KeyError. + if 'sort_function' in kwargs: + kwargs['cmp'] = kwargs.pop('sort_function') + self._values.sort(*args, **kwargs) class RepeatedScalarFieldContainer(BaseContainer): @@ -235,6 +240,11 @@ class RepeatedCompositeFieldContainer(BaseContainer): """ self.extend(other._values) + def remove(self, elem): + """Removes an item from the list. Similar to list.remove().""" + self._values.remove(elem) + self._message_listener.Modified() + def __getslice__(self, start, stop): """Retrieves the subset of items from between the specified indices.""" return self._values[start:stop] diff --git a/third_party/protobuf/python/google/protobuf/internal/cpp_message.py b/third_party/protobuf/python/google/protobuf/internal/cpp_message.py index 3f42650..23ab9ba 100644 --- a/third_party/protobuf/python/google/protobuf/internal/cpp_message.py +++ b/third_party/protobuf/python/google/protobuf/internal/cpp_message.py @@ -34,8 +34,10 @@ Descriptor objects at runtime backed by the protocol buffer C++ API. __author__ = 'petar@google.com (Petar Petrov)' +import copy_reg import operator from google.protobuf.internal import _net_proto2___python +from google.protobuf.internal import enum_type_wrapper from google.protobuf import message @@ -156,10 +158,12 @@ class RepeatedScalarContainer(object): def __hash__(self): raise TypeError('unhashable object') - def sort(self, sort_function=cmp): - values = self[slice(None, None, None)] - values.sort(sort_function) - self._cmsg.AssignRepeatedScalar(self._cfield_descriptor, values) + def sort(self, *args, **kwargs): + # Maintain compatibility with the previous interface. + if 'sort_function' in kwargs: + kwargs['cmp'] = kwargs.pop('sort_function') + self._cmsg.AssignRepeatedScalar(self._cfield_descriptor, + sorted(self, *args, **kwargs)) def RepeatedScalarProperty(cdescriptor): @@ -202,6 +206,12 @@ class RepeatedCompositeContainer(object): for message in elem_seq: self.add().MergeFrom(message) + def remove(self, value): + # TODO(protocol-devel): This is inefficient as it needs to generate a + # message pointer for each message only to do index(). Move this to a C++ + # extension function. + self.__delitem__(self[slice(None, None, None)].index(value)) + def MergeFrom(self, other): for message in other[:]: self.add().MergeFrom(message) @@ -236,27 +246,29 @@ class RepeatedCompositeContainer(object): def __hash__(self): raise TypeError('unhashable object') - def sort(self, sort_function=cmp): - messages = [] - for index in range(len(self)): - # messages[i][0] is where the i-th element of the new array has to come - # from. - # messages[i][1] is where the i-th element of the old array has to go. - messages.append([index, 0, self[index]]) - messages.sort(lambda x,y: sort_function(x[2], y[2])) + def sort(self, cmp=None, key=None, reverse=False, **kwargs): + # Maintain compatibility with the old interface. + if cmp is None and 'sort_function' in kwargs: + cmp = kwargs.pop('sort_function') - # Remember which position each elements has to move to. - for i in range(len(messages)): - messages[messages[i][0]][1] = i + # The cmp function, if provided, is passed the results of the key function, + # so we only need to wrap one of them. + if key is None: + index_key = self.__getitem__ + else: + index_key = lambda i: key(self[i]) + + # Sort the list of current indexes by the underlying object. + indexes = range(len(self)) + indexes.sort(cmp=cmp, key=index_key, reverse=reverse) # Apply the transposition. - for i in range(len(messages)): - from_position = messages[i][0] - if i == from_position: + for dest, src in enumerate(indexes): + if dest == src: continue - self._cmsg.SwapRepeatedFieldElements( - self._cfield_descriptor, i, from_position) - messages[messages[i][1]][0] = from_position + self._cmsg.SwapRepeatedFieldElements(self._cfield_descriptor, dest, src) + # Don't swap the same value twice. + indexes[src] = src def RepeatedCompositeProperty(cdescriptor, message_type): @@ -359,11 +371,12 @@ class ExtensionDict(object): return None -def NewMessage(message_descriptor, dictionary): +def NewMessage(bases, message_descriptor, dictionary): """Creates a new protocol message *class*.""" _AddClassAttributesForNestedExtensions(message_descriptor, dictionary) _AddEnumValues(message_descriptor, dictionary) _AddDescriptors(message_descriptor, dictionary) + return bases def InitMessage(message_descriptor, cls): @@ -372,6 +385,7 @@ def InitMessage(message_descriptor, cls): _AddInitMethod(message_descriptor, cls) _AddMessageMethods(message_descriptor, cls) _AddPropertiesForExtensions(message_descriptor, cls) + copy_reg.pickle(cls, lambda obj: (cls, (), obj.__getstate__())) def _AddDescriptors(message_descriptor, dictionary): @@ -387,7 +401,7 @@ def _AddDescriptors(message_descriptor, dictionary): field.full_name) dictionary['__slots__'] = list(dictionary['__descriptors'].iterkeys()) + [ - '_cmsg', '_owner', '_composite_fields', 'Extensions'] + '_cmsg', '_owner', '_composite_fields', 'Extensions', '_HACK_REFCOUNTS'] def _AddEnumValues(message_descriptor, dictionary): @@ -398,6 +412,7 @@ def _AddEnumValues(message_descriptor, dictionary): dictionary: Class dictionary that should be populated. """ for enum_type in message_descriptor.enum_types: + dictionary[enum_type.name] = enum_type_wrapper.EnumTypeWrapper(enum_type) for enum_value in enum_type.values: dictionary[enum_value.name] = enum_value.number @@ -439,28 +454,35 @@ def _AddInitMethod(message_descriptor, cls): def Init(self, **kwargs): """Message constructor.""" cmessage = kwargs.pop('__cmessage', None) - if cmessage is None: - self._cmsg = NewCMessage(message_descriptor.full_name) - else: + if cmessage: self._cmsg = cmessage + else: + self._cmsg = NewCMessage(message_descriptor.full_name) # Keep a reference to the owner, as the owner keeps a reference to the # underlying protocol buffer message. owner = kwargs.pop('__owner', None) - if owner is not None: + if owner: self._owner = owner - self.Extensions = ExtensionDict(self) + if message_descriptor.is_extendable: + self.Extensions = ExtensionDict(self) + else: + # Reference counting in the C++ code is broken and depends on + # the Extensions reference to keep this object alive during unit + # tests (see b/4856052). Remove this once b/4945904 is fixed. + self._HACK_REFCOUNTS = self self._composite_fields = {} for field_name, field_value in kwargs.iteritems(): field_cdescriptor = self.__descriptors.get(field_name, None) - if field_cdescriptor is None: + if not field_cdescriptor: raise ValueError('Protocol message has no "%s" field.' % field_name) if field_cdescriptor.label == _LABEL_REPEATED: if field_cdescriptor.cpp_type == _CPPTYPE_MESSAGE: + field_name = getattr(self, field_name) for val in field_value: - getattr(self, field_name).add().MergeFrom(val) + field_name.add().MergeFrom(val) else: getattr(self, field_name).extend(field_value) elif field_cdescriptor.cpp_type == _CPPTYPE_MESSAGE: @@ -497,12 +519,34 @@ def _AddMessageMethods(message_descriptor, cls): return self._cmsg.HasField(field_name) def ClearField(self, field_name): + child_cmessage = None if field_name in self._composite_fields: + child_field = self._composite_fields[field_name] del self._composite_fields[field_name] - self._cmsg.ClearField(field_name) + + child_cdescriptor = self.__descriptors[field_name] + # TODO(anuraag): Support clearing repeated message fields as well. + if (child_cdescriptor.label != _LABEL_REPEATED and + child_cdescriptor.cpp_type == _CPPTYPE_MESSAGE): + child_field._owner = None + child_cmessage = child_field._cmsg + + if child_cmessage is not None: + self._cmsg.ClearField(field_name, child_cmessage) + else: + self._cmsg.ClearField(field_name) def Clear(self): - return self._cmsg.Clear() + cmessages_to_release = [] + for field_name, child_field in self._composite_fields.iteritems(): + child_cdescriptor = self.__descriptors[field_name] + # TODO(anuraag): Support clearing repeated message fields as well. + if (child_cdescriptor.label != _LABEL_REPEATED and + child_cdescriptor.cpp_type == _CPPTYPE_MESSAGE): + child_field._owner = None + cmessages_to_release.append((child_cdescriptor, child_field._cmsg)) + self._composite_fields.clear() + self._cmsg.Clear(cmessages_to_release) def IsInitialized(self, errors=None): if self._cmsg.IsInitialized(): @@ -514,8 +558,8 @@ def _AddMessageMethods(message_descriptor, cls): def SerializeToString(self): if not self.IsInitialized(): raise message.EncodeError( - 'Message is missing required fields: ' + - ','.join(self.FindInitializationErrors())) + 'Message %s is missing required fields: %s' % ( + self._cmsg.full_name, ','.join(self.FindInitializationErrors()))) return self._cmsg.SerializeToString() def SerializePartialToString(self): @@ -534,7 +578,8 @@ def _AddMessageMethods(message_descriptor, cls): def MergeFrom(self, msg): if not isinstance(msg, cls): raise TypeError( - "Parameter to MergeFrom() must be instance of same class.") + "Parameter to MergeFrom() must be instance of same class: " + "expected %s got %s." % (cls.__name__, type(msg).__name__)) self._cmsg.MergeFrom(msg._cmsg) def CopyFrom(self, msg): @@ -581,6 +626,8 @@ def _AddMessageMethods(message_descriptor, cls): raise TypeError('unhashable object') def __unicode__(self): + # Lazy import to prevent circular import when text_format imports this file. + from google.protobuf import text_format return text_format.MessageToString(self, as_utf8=True).decode('utf-8') # Attach the local methods to the message class. diff --git a/third_party/protobuf/python/google/protobuf/internal/decoder.py b/third_party/protobuf/python/google/protobuf/internal/decoder.py index 55f746f..cb6f572 100755 --- a/third_party/protobuf/python/google/protobuf/internal/decoder.py +++ b/third_party/protobuf/python/google/protobuf/internal/decoder.py @@ -576,6 +576,7 @@ def MessageSetItemDecoder(extensions_by_number): local_SkipField = SkipField def DecodeItem(buffer, pos, end, message, field_dict): + message_set_item_start = pos type_id = -1 message_start = -1 message_end = -1 @@ -614,6 +615,11 @@ def MessageSetItemDecoder(extensions_by_number): # The only reason _InternalParse would return early is if it encountered # an end-group tag. raise _DecodeError('Unexpected end-group tag.') + else: + if not message._unknown_fields: + message._unknown_fields = [] + message._unknown_fields.append((MESSAGE_SET_ITEM_TAG, + buffer[message_set_item_start:pos])) return pos diff --git a/third_party/protobuf/python/google/protobuf/internal/descriptor_database_test.py b/third_party/protobuf/python/google/protobuf/internal/descriptor_database_test.py new file mode 100644 index 0000000..d0ca789 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/descriptor_database_test.py @@ -0,0 +1,63 @@ +#! /usr/bin/python +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for google.protobuf.descriptor_database.""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + +import unittest +from google.protobuf import descriptor_pb2 +from google.protobuf.internal import factory_test2_pb2 +from google.protobuf import descriptor_database + + +class DescriptorDatabaseTest(unittest.TestCase): + + def testAdd(self): + db = descriptor_database.DescriptorDatabase() + file_desc_proto = descriptor_pb2.FileDescriptorProto.FromString( + factory_test2_pb2.DESCRIPTOR.serialized_pb) + db.Add(file_desc_proto) + + self.assertEquals(file_desc_proto, db.FindFileByName( + 'net/proto2/python/internal/factory_test2.proto')) + self.assertEquals(file_desc_proto, db.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory2Message')) + self.assertEquals(file_desc_proto, db.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory2Message.NestedFactory2Message')) + self.assertEquals(file_desc_proto, db.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory2Enum')) + self.assertEquals(file_desc_proto, db.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory2Message.NestedFactory2Enum')) + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/descriptor_pool_test.py b/third_party/protobuf/python/google/protobuf/internal/descriptor_pool_test.py new file mode 100644 index 0000000..a615d78 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/descriptor_pool_test.py @@ -0,0 +1,220 @@ +#! /usr/bin/python +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for google.protobuf.descriptor_pool.""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + +import unittest +from google.protobuf import descriptor_pb2 +from google.protobuf.internal import factory_test1_pb2 +from google.protobuf.internal import factory_test2_pb2 +from google.protobuf import descriptor +from google.protobuf import descriptor_database +from google.protobuf import descriptor_pool + + +class DescriptorPoolTest(unittest.TestCase): + + def setUp(self): + self.pool = descriptor_pool.DescriptorPool() + self.factory_test1_fd = descriptor_pb2.FileDescriptorProto.FromString( + factory_test1_pb2.DESCRIPTOR.serialized_pb) + self.factory_test2_fd = descriptor_pb2.FileDescriptorProto.FromString( + factory_test2_pb2.DESCRIPTOR.serialized_pb) + self.pool.Add(self.factory_test1_fd) + self.pool.Add(self.factory_test2_fd) + + def testFindFileByName(self): + name1 = 'net/proto2/python/internal/factory_test1.proto' + file_desc1 = self.pool.FindFileByName(name1) + self.assertIsInstance(file_desc1, descriptor.FileDescriptor) + self.assertEquals(name1, file_desc1.name) + self.assertEquals('net.proto2.python.internal', file_desc1.package) + self.assertIn('Factory1Message', file_desc1.message_types_by_name) + + name2 = 'net/proto2/python/internal/factory_test2.proto' + file_desc2 = self.pool.FindFileByName(name2) + self.assertIsInstance(file_desc2, descriptor.FileDescriptor) + self.assertEquals(name2, file_desc2.name) + self.assertEquals('net.proto2.python.internal', file_desc2.package) + self.assertIn('Factory2Message', file_desc2.message_types_by_name) + + def testFindFileByNameFailure(self): + try: + self.pool.FindFileByName('Does not exist') + self.fail('Expected KeyError') + except KeyError: + pass + + def testFindFileContainingSymbol(self): + file_desc1 = self.pool.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory1Message') + self.assertIsInstance(file_desc1, descriptor.FileDescriptor) + self.assertEquals('net/proto2/python/internal/factory_test1.proto', + file_desc1.name) + self.assertEquals('net.proto2.python.internal', file_desc1.package) + self.assertIn('Factory1Message', file_desc1.message_types_by_name) + + file_desc2 = self.pool.FindFileContainingSymbol( + 'net.proto2.python.internal.Factory2Message') + self.assertIsInstance(file_desc2, descriptor.FileDescriptor) + self.assertEquals('net/proto2/python/internal/factory_test2.proto', + file_desc2.name) + self.assertEquals('net.proto2.python.internal', file_desc2.package) + self.assertIn('Factory2Message', file_desc2.message_types_by_name) + + def testFindFileContainingSymbolFailure(self): + try: + self.pool.FindFileContainingSymbol('Does not exist') + self.fail('Expected KeyError') + except KeyError: + pass + + def testFindMessageTypeByName(self): + msg1 = self.pool.FindMessageTypeByName( + 'net.proto2.python.internal.Factory1Message') + self.assertIsInstance(msg1, descriptor.Descriptor) + self.assertEquals('Factory1Message', msg1.name) + self.assertEquals('net.proto2.python.internal.Factory1Message', + msg1.full_name) + self.assertEquals(None, msg1.containing_type) + + nested_msg1 = msg1.nested_types[0] + self.assertEquals('NestedFactory1Message', nested_msg1.name) + self.assertEquals(msg1, nested_msg1.containing_type) + + nested_enum1 = msg1.enum_types[0] + self.assertEquals('NestedFactory1Enum', nested_enum1.name) + self.assertEquals(msg1, nested_enum1.containing_type) + + self.assertEquals(nested_msg1, msg1.fields_by_name[ + 'nested_factory_1_message'].message_type) + self.assertEquals(nested_enum1, msg1.fields_by_name[ + 'nested_factory_1_enum'].enum_type) + + msg2 = self.pool.FindMessageTypeByName( + 'net.proto2.python.internal.Factory2Message') + self.assertIsInstance(msg2, descriptor.Descriptor) + self.assertEquals('Factory2Message', msg2.name) + self.assertEquals('net.proto2.python.internal.Factory2Message', + msg2.full_name) + self.assertIsNone(msg2.containing_type) + + nested_msg2 = msg2.nested_types[0] + self.assertEquals('NestedFactory2Message', nested_msg2.name) + self.assertEquals(msg2, nested_msg2.containing_type) + + nested_enum2 = msg2.enum_types[0] + self.assertEquals('NestedFactory2Enum', nested_enum2.name) + self.assertEquals(msg2, nested_enum2.containing_type) + + self.assertEquals(nested_msg2, msg2.fields_by_name[ + 'nested_factory_2_message'].message_type) + self.assertEquals(nested_enum2, msg2.fields_by_name[ + 'nested_factory_2_enum'].enum_type) + + self.assertTrue(msg2.fields_by_name['int_with_default'].has_default) + self.assertEquals( + 1776, msg2.fields_by_name['int_with_default'].default_value) + + self.assertTrue(msg2.fields_by_name['double_with_default'].has_default) + self.assertEquals( + 9.99, msg2.fields_by_name['double_with_default'].default_value) + + self.assertTrue(msg2.fields_by_name['string_with_default'].has_default) + self.assertEquals( + 'hello world', msg2.fields_by_name['string_with_default'].default_value) + + self.assertTrue(msg2.fields_by_name['bool_with_default'].has_default) + self.assertFalse(msg2.fields_by_name['bool_with_default'].default_value) + + self.assertTrue(msg2.fields_by_name['enum_with_default'].has_default) + self.assertEquals( + 1, msg2.fields_by_name['enum_with_default'].default_value) + + msg3 = self.pool.FindMessageTypeByName( + 'net.proto2.python.internal.Factory2Message.NestedFactory2Message') + self.assertEquals(nested_msg2, msg3) + + def testFindMessageTypeByNameFailure(self): + try: + self.pool.FindMessageTypeByName('Does not exist') + self.fail('Expected KeyError') + except KeyError: + pass + + def testFindEnumTypeByName(self): + enum1 = self.pool.FindEnumTypeByName( + 'net.proto2.python.internal.Factory1Enum') + self.assertIsInstance(enum1, descriptor.EnumDescriptor) + self.assertEquals(0, enum1.values_by_name['FACTORY_1_VALUE_0'].number) + self.assertEquals(1, enum1.values_by_name['FACTORY_1_VALUE_1'].number) + + nested_enum1 = self.pool.FindEnumTypeByName( + 'net.proto2.python.internal.Factory1Message.NestedFactory1Enum') + self.assertIsInstance(nested_enum1, descriptor.EnumDescriptor) + self.assertEquals( + 0, nested_enum1.values_by_name['NESTED_FACTORY_1_VALUE_0'].number) + self.assertEquals( + 1, nested_enum1.values_by_name['NESTED_FACTORY_1_VALUE_1'].number) + + enum2 = self.pool.FindEnumTypeByName( + 'net.proto2.python.internal.Factory2Enum') + self.assertIsInstance(enum2, descriptor.EnumDescriptor) + self.assertEquals(0, enum2.values_by_name['FACTORY_2_VALUE_0'].number) + self.assertEquals(1, enum2.values_by_name['FACTORY_2_VALUE_1'].number) + + nested_enum2 = self.pool.FindEnumTypeByName( + 'net.proto2.python.internal.Factory2Message.NestedFactory2Enum') + self.assertIsInstance(nested_enum2, descriptor.EnumDescriptor) + self.assertEquals( + 0, nested_enum2.values_by_name['NESTED_FACTORY_2_VALUE_0'].number) + self.assertEquals( + 1, nested_enum2.values_by_name['NESTED_FACTORY_2_VALUE_1'].number) + + def testFindEnumTypeByNameFailure(self): + try: + self.pool.FindEnumTypeByName('Does not exist') + self.fail('Expected KeyError') + except KeyError: + pass + + def testUserDefinedDB(self): + db = descriptor_database.DescriptorDatabase() + self.pool = descriptor_pool.DescriptorPool(db) + db.Add(self.factory_test1_fd) + db.Add(self.factory_test2_fd) + self.testFindMessageTypeByName() + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/descriptor_test.py b/third_party/protobuf/python/google/protobuf/internal/descriptor_test.py index 05c2745..c74f882 100755 --- a/third_party/protobuf/python/google/protobuf/internal/descriptor_test.py +++ b/third_party/protobuf/python/google/protobuf/internal/descriptor_test.py @@ -35,6 +35,7 @@ __author__ = 'robinson@google.com (Will Robinson)' import unittest +from google.protobuf import unittest_custom_options_pb2 from google.protobuf import unittest_import_pb2 from google.protobuf import unittest_pb2 from google.protobuf import descriptor_pb2 @@ -101,6 +102,15 @@ class DescriptorTest(unittest.TestCase): self.my_method ]) + def testEnumValueName(self): + self.assertEqual(self.my_message.EnumValueName('ForeignEnum', 4), + 'FOREIGN_FOO') + + self.assertEqual( + self.my_message.enum_types_by_name[ + 'ForeignEnum'].values_by_number[4].name, + self.my_message.EnumValueName('ForeignEnum', 4)) + def testEnumFixups(self): self.assertEqual(self.my_enum, self.my_enum.values[0].type) @@ -125,6 +135,257 @@ class DescriptorTest(unittest.TestCase): self.assertEqual(self.my_service.GetOptions(), descriptor_pb2.ServiceOptions()) + def testSimpleCustomOptions(self): + file_descriptor = unittest_custom_options_pb2.DESCRIPTOR + message_descriptor =\ + unittest_custom_options_pb2.TestMessageWithCustomOptions.DESCRIPTOR + field_descriptor = message_descriptor.fields_by_name["field1"] + enum_descriptor = message_descriptor.enum_types_by_name["AnEnum"] + enum_value_descriptor =\ + message_descriptor.enum_values_by_name["ANENUM_VAL2"] + service_descriptor =\ + unittest_custom_options_pb2.TestServiceWithCustomOptions.DESCRIPTOR + method_descriptor = service_descriptor.FindMethodByName("Foo") + + file_options = file_descriptor.GetOptions() + file_opt1 = unittest_custom_options_pb2.file_opt1 + self.assertEqual(9876543210, file_options.Extensions[file_opt1]) + message_options = message_descriptor.GetOptions() + message_opt1 = unittest_custom_options_pb2.message_opt1 + self.assertEqual(-56, message_options.Extensions[message_opt1]) + field_options = field_descriptor.GetOptions() + field_opt1 = unittest_custom_options_pb2.field_opt1 + self.assertEqual(8765432109, field_options.Extensions[field_opt1]) + field_opt2 = unittest_custom_options_pb2.field_opt2 + self.assertEqual(42, field_options.Extensions[field_opt2]) + enum_options = enum_descriptor.GetOptions() + enum_opt1 = unittest_custom_options_pb2.enum_opt1 + self.assertEqual(-789, enum_options.Extensions[enum_opt1]) + enum_value_options = enum_value_descriptor.GetOptions() + enum_value_opt1 = unittest_custom_options_pb2.enum_value_opt1 + self.assertEqual(123, enum_value_options.Extensions[enum_value_opt1]) + + service_options = service_descriptor.GetOptions() + service_opt1 = unittest_custom_options_pb2.service_opt1 + self.assertEqual(-9876543210, service_options.Extensions[service_opt1]) + method_options = method_descriptor.GetOptions() + method_opt1 = unittest_custom_options_pb2.method_opt1 + self.assertEqual(unittest_custom_options_pb2.METHODOPT1_VAL2, + method_options.Extensions[method_opt1]) + + def testDifferentCustomOptionTypes(self): + kint32min = -2**31 + kint64min = -2**63 + kint32max = 2**31 - 1 + kint64max = 2**63 - 1 + kuint32max = 2**32 - 1 + kuint64max = 2**64 - 1 + + message_descriptor =\ + unittest_custom_options_pb2.CustomOptionMinIntegerValues.DESCRIPTOR + message_options = message_descriptor.GetOptions() + self.assertEqual(False, message_options.Extensions[ + unittest_custom_options_pb2.bool_opt]) + self.assertEqual(kint32min, message_options.Extensions[ + unittest_custom_options_pb2.int32_opt]) + self.assertEqual(kint64min, message_options.Extensions[ + unittest_custom_options_pb2.int64_opt]) + self.assertEqual(0, message_options.Extensions[ + unittest_custom_options_pb2.uint32_opt]) + self.assertEqual(0, message_options.Extensions[ + unittest_custom_options_pb2.uint64_opt]) + self.assertEqual(kint32min, message_options.Extensions[ + unittest_custom_options_pb2.sint32_opt]) + self.assertEqual(kint64min, message_options.Extensions[ + unittest_custom_options_pb2.sint64_opt]) + self.assertEqual(0, message_options.Extensions[ + unittest_custom_options_pb2.fixed32_opt]) + self.assertEqual(0, message_options.Extensions[ + unittest_custom_options_pb2.fixed64_opt]) + self.assertEqual(kint32min, message_options.Extensions[ + unittest_custom_options_pb2.sfixed32_opt]) + self.assertEqual(kint64min, message_options.Extensions[ + unittest_custom_options_pb2.sfixed64_opt]) + + message_descriptor =\ + unittest_custom_options_pb2.CustomOptionMaxIntegerValues.DESCRIPTOR + message_options = message_descriptor.GetOptions() + self.assertEqual(True, message_options.Extensions[ + unittest_custom_options_pb2.bool_opt]) + self.assertEqual(kint32max, message_options.Extensions[ + unittest_custom_options_pb2.int32_opt]) + self.assertEqual(kint64max, message_options.Extensions[ + unittest_custom_options_pb2.int64_opt]) + self.assertEqual(kuint32max, message_options.Extensions[ + unittest_custom_options_pb2.uint32_opt]) + self.assertEqual(kuint64max, message_options.Extensions[ + unittest_custom_options_pb2.uint64_opt]) + self.assertEqual(kint32max, message_options.Extensions[ + unittest_custom_options_pb2.sint32_opt]) + self.assertEqual(kint64max, message_options.Extensions[ + unittest_custom_options_pb2.sint64_opt]) + self.assertEqual(kuint32max, message_options.Extensions[ + unittest_custom_options_pb2.fixed32_opt]) + self.assertEqual(kuint64max, message_options.Extensions[ + unittest_custom_options_pb2.fixed64_opt]) + self.assertEqual(kint32max, message_options.Extensions[ + unittest_custom_options_pb2.sfixed32_opt]) + self.assertEqual(kint64max, message_options.Extensions[ + unittest_custom_options_pb2.sfixed64_opt]) + + message_descriptor =\ + unittest_custom_options_pb2.CustomOptionOtherValues.DESCRIPTOR + message_options = message_descriptor.GetOptions() + self.assertEqual(-100, message_options.Extensions[ + unittest_custom_options_pb2.int32_opt]) + self.assertAlmostEqual(12.3456789, message_options.Extensions[ + unittest_custom_options_pb2.float_opt], 6) + self.assertAlmostEqual(1.234567890123456789, message_options.Extensions[ + unittest_custom_options_pb2.double_opt]) + self.assertEqual("Hello, \"World\"", message_options.Extensions[ + unittest_custom_options_pb2.string_opt]) + self.assertEqual("Hello\0World", message_options.Extensions[ + unittest_custom_options_pb2.bytes_opt]) + dummy_enum = unittest_custom_options_pb2.DummyMessageContainingEnum + self.assertEqual( + dummy_enum.TEST_OPTION_ENUM_TYPE2, + message_options.Extensions[unittest_custom_options_pb2.enum_opt]) + + message_descriptor =\ + unittest_custom_options_pb2.SettingRealsFromPositiveInts.DESCRIPTOR + message_options = message_descriptor.GetOptions() + self.assertAlmostEqual(12, message_options.Extensions[ + unittest_custom_options_pb2.float_opt], 6) + self.assertAlmostEqual(154, message_options.Extensions[ + unittest_custom_options_pb2.double_opt]) + + message_descriptor =\ + unittest_custom_options_pb2.SettingRealsFromNegativeInts.DESCRIPTOR + message_options = message_descriptor.GetOptions() + self.assertAlmostEqual(-12, message_options.Extensions[ + unittest_custom_options_pb2.float_opt], 6) + self.assertAlmostEqual(-154, message_options.Extensions[ + unittest_custom_options_pb2.double_opt]) + + def testComplexExtensionOptions(self): + descriptor =\ + unittest_custom_options_pb2.VariousComplexOptions.DESCRIPTOR + options = descriptor.GetOptions() + self.assertEqual(42, options.Extensions[ + unittest_custom_options_pb2.complex_opt1].foo) + self.assertEqual(324, options.Extensions[ + unittest_custom_options_pb2.complex_opt1].Extensions[ + unittest_custom_options_pb2.quux]) + self.assertEqual(876, options.Extensions[ + unittest_custom_options_pb2.complex_opt1].Extensions[ + unittest_custom_options_pb2.corge].qux) + self.assertEqual(987, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].baz) + self.assertEqual(654, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].Extensions[ + unittest_custom_options_pb2.grault]) + self.assertEqual(743, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].bar.foo) + self.assertEqual(1999, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].bar.Extensions[ + unittest_custom_options_pb2.quux]) + self.assertEqual(2008, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].bar.Extensions[ + unittest_custom_options_pb2.corge].qux) + self.assertEqual(741, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].Extensions[ + unittest_custom_options_pb2.garply].foo) + self.assertEqual(1998, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].Extensions[ + unittest_custom_options_pb2.garply].Extensions[ + unittest_custom_options_pb2.quux]) + self.assertEqual(2121, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].Extensions[ + unittest_custom_options_pb2.garply].Extensions[ + unittest_custom_options_pb2.corge].qux) + self.assertEqual(1971, options.Extensions[ + unittest_custom_options_pb2.ComplexOptionType2 + .ComplexOptionType4.complex_opt4].waldo) + self.assertEqual(321, options.Extensions[ + unittest_custom_options_pb2.complex_opt2].fred.waldo) + self.assertEqual(9, options.Extensions[ + unittest_custom_options_pb2.complex_opt3].qux) + self.assertEqual(22, options.Extensions[ + unittest_custom_options_pb2.complex_opt3].complexoptiontype5.plugh) + self.assertEqual(24, options.Extensions[ + unittest_custom_options_pb2.complexopt6].xyzzy) + + # Check that aggregate options were parsed and saved correctly in + # the appropriate descriptors. + def testAggregateOptions(self): + file_descriptor = unittest_custom_options_pb2.DESCRIPTOR + message_descriptor =\ + unittest_custom_options_pb2.AggregateMessage.DESCRIPTOR + field_descriptor = message_descriptor.fields_by_name["fieldname"] + enum_descriptor = unittest_custom_options_pb2.AggregateEnum.DESCRIPTOR + enum_value_descriptor = enum_descriptor.values_by_name["VALUE"] + service_descriptor =\ + unittest_custom_options_pb2.AggregateService.DESCRIPTOR + method_descriptor = service_descriptor.FindMethodByName("Method") + + # Tests for the different types of data embedded in fileopt + file_options = file_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.fileopt] + self.assertEqual(100, file_options.i) + self.assertEqual("FileAnnotation", file_options.s) + self.assertEqual("NestedFileAnnotation", file_options.sub.s) + self.assertEqual("FileExtensionAnnotation", file_options.file.Extensions[ + unittest_custom_options_pb2.fileopt].s) + self.assertEqual("EmbeddedMessageSetElement", file_options.mset.Extensions[ + unittest_custom_options_pb2.AggregateMessageSetElement + .message_set_extension].s) + + # Simple tests for all the other types of annotations + self.assertEqual( + "MessageAnnotation", + message_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.msgopt].s) + self.assertEqual( + "FieldAnnotation", + field_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.fieldopt].s) + self.assertEqual( + "EnumAnnotation", + enum_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.enumopt].s) + self.assertEqual( + "EnumValueAnnotation", + enum_value_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.enumvalopt].s) + self.assertEqual( + "ServiceAnnotation", + service_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.serviceopt].s) + self.assertEqual( + "MethodAnnotation", + method_descriptor.GetOptions().Extensions[ + unittest_custom_options_pb2.methodopt].s) + + def testNestedOptions(self): + nested_message =\ + unittest_custom_options_pb2.NestedOptionType.NestedMessage.DESCRIPTOR + self.assertEqual(1001, nested_message.GetOptions().Extensions[ + unittest_custom_options_pb2.message_opt1]) + nested_field = nested_message.fields_by_name["nested_field"] + self.assertEqual(1002, nested_field.GetOptions().Extensions[ + unittest_custom_options_pb2.field_opt1]) + outer_message =\ + unittest_custom_options_pb2.NestedOptionType.DESCRIPTOR + nested_enum = outer_message.enum_types_by_name["NestedEnum"] + self.assertEqual(1003, nested_enum.GetOptions().Extensions[ + unittest_custom_options_pb2.enum_opt1]) + nested_enum_value = outer_message.enum_values_by_name["NESTED_ENUM_VALUE"] + self.assertEqual(1004, nested_enum_value.GetOptions().Extensions[ + unittest_custom_options_pb2.enum_value_opt1]) + nested_extension = outer_message.extensions_by_name["nested_extension"] + self.assertEqual(1005, nested_extension.GetOptions().Extensions[ + unittest_custom_options_pb2.field_opt2]) + def testFileDescriptorReferences(self): self.assertEqual(self.my_enum.file, self.my_file) self.assertEqual(self.my_message.file, self.my_file) @@ -273,6 +534,7 @@ class DescriptorCopyToProtoTest(unittest.TestCase): UNITTEST_IMPORT_FILE_DESCRIPTOR_ASCII = (""" name: 'google/protobuf/unittest_import.proto' package: 'protobuf_unittest_import' + dependency: 'google/protobuf/unittest_import_public.proto' message_type: < name: 'ImportMessage' field: < @@ -302,6 +564,7 @@ class DescriptorCopyToProtoTest(unittest.TestCase): java_package: 'com.google.protobuf.test' optimize_for: 1 # SPEED > + public_dependency: 0 """) self._InternalTestCopyToProto( @@ -330,5 +593,21 @@ class DescriptorCopyToProtoTest(unittest.TestCase): TEST_SERVICE_ASCII) +class MakeDescriptorTest(unittest.TestCase): + def testMakeDescriptorWithUnsignedIntField(self): + file_descriptor_proto = descriptor_pb2.FileDescriptorProto() + file_descriptor_proto.name = 'Foo' + message_type = file_descriptor_proto.message_type.add() + message_type.name = file_descriptor_proto.name + field = message_type.field.add() + field.number = 1 + field.name = 'uint64_field' + field.label = descriptor.FieldDescriptor.LABEL_REQUIRED + field.type = descriptor.FieldDescriptor.TYPE_UINT64 + result = descriptor.MakeDescriptor(message_type) + self.assertEqual(result.fields[0].cpp_type, + descriptor.FieldDescriptor.CPPTYPE_UINT64) + + if __name__ == '__main__': unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/enum_type_wrapper.py b/third_party/protobuf/python/google/protobuf/internal/enum_type_wrapper.py new file mode 100644 index 0000000..7b28645 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/enum_type_wrapper.py @@ -0,0 +1,89 @@ +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""A simple wrapper around enum types to expose utility functions. + +Instances are created as properties with the same name as the enum they wrap +on proto classes. For usage, see: + reflection_test.py +""" + +__author__ = 'rabsatt@google.com (Kevin Rabsatt)' + + +class EnumTypeWrapper(object): + """A utility for finding the names of enum values.""" + + DESCRIPTOR = None + + def __init__(self, enum_type): + """Inits EnumTypeWrapper with an EnumDescriptor.""" + self._enum_type = enum_type + self.DESCRIPTOR = enum_type; + + def Name(self, number): + """Returns a string containing the name of an enum value.""" + if number in self._enum_type.values_by_number: + return self._enum_type.values_by_number[number].name + raise ValueError('Enum %s has no name defined for value %d' % ( + self._enum_type.name, number)) + + def Value(self, name): + """Returns the value coresponding to the given enum name.""" + if name in self._enum_type.values_by_name: + return self._enum_type.values_by_name[name].number + raise ValueError('Enum %s has no value defined for name %s' % ( + self._enum_type.name, name)) + + def keys(self): + """Return a list of the string names in the enum. + + These are returned in the order they were defined in the .proto file. + """ + + return [value_descriptor.name + for value_descriptor in self._enum_type.values] + + def values(self): + """Return a list of the integer values in the enum. + + These are returned in the order they were defined in the .proto file. + """ + + return [value_descriptor.number + for value_descriptor in self._enum_type.values] + + def items(self): + """Return a list of the (name, value) pairs of the enum. + + These are returned in the order they were defined in the .proto file. + """ + return [(value_descriptor.name, value_descriptor.number) + for value_descriptor in self._enum_type.values] diff --git a/third_party/protobuf/python/google/protobuf/internal/factory_test1.proto b/third_party/protobuf/python/google/protobuf/internal/factory_test1.proto new file mode 100644 index 0000000..9f55e03 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/factory_test1.proto @@ -0,0 +1,55 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: matthewtoia@google.com (Matt Toia) + + +package google.protobuf.python.internal; + + +enum Factory1Enum { + FACTORY_1_VALUE_0 = 0; + FACTORY_1_VALUE_1 = 1; +} + +message Factory1Message { + optional Factory1Enum factory_1_enum = 1; + enum NestedFactory1Enum { + NESTED_FACTORY_1_VALUE_0 = 0; + NESTED_FACTORY_1_VALUE_1 = 1; + } + optional NestedFactory1Enum nested_factory_1_enum = 2; + message NestedFactory1Message { + optional string value = 1; + } + optional NestedFactory1Message nested_factory_1_message = 3; + optional int32 scalar_value = 4; + repeated string list_value = 5; +} diff --git a/third_party/protobuf/python/google/protobuf/internal/factory_test2.proto b/third_party/protobuf/python/google/protobuf/internal/factory_test2.proto new file mode 100644 index 0000000..d3ce4d7 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/factory_test2.proto @@ -0,0 +1,77 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: matthewtoia@google.com (Matt Toia) + + +package google.protobuf.python.internal; + +import "google/protobuf/internal/factory_test1.proto"; + + +enum Factory2Enum { + FACTORY_2_VALUE_0 = 0; + FACTORY_2_VALUE_1 = 1; +} + +message Factory2Message { + required int32 mandatory = 1; + optional Factory2Enum factory_2_enum = 2; + enum NestedFactory2Enum { + NESTED_FACTORY_2_VALUE_0 = 0; + NESTED_FACTORY_2_VALUE_1 = 1; + } + optional NestedFactory2Enum nested_factory_2_enum = 3; + message NestedFactory2Message { + optional string value = 1; + } + optional NestedFactory2Message nested_factory_2_message = 4; + optional Factory1Message factory_1_message = 5; + optional Factory1Enum factory_1_enum = 6; + optional Factory1Message.NestedFactory1Enum nested_factory_1_enum = 7; + optional Factory1Message.NestedFactory1Message nested_factory_1_message = 8; + optional Factory2Message circular_message = 9; + optional string scalar_value = 10; + repeated string list_value = 11; + repeated group Grouped = 12 { + optional string part_1 = 13; + optional string part_2 = 14; + } + optional LoopMessage loop = 15; + optional int32 int_with_default = 16 [default = 1776]; + optional double double_with_default = 17 [default = 9.99]; + optional string string_with_default = 18 [default = "hello world"]; + optional bool bool_with_default = 19 [default = false]; + optional Factory2Enum enum_with_default = 20 [default = FACTORY_2_VALUE_1]; +} + +message LoopMessage { + optional Factory2Message loop = 1; +} diff --git a/third_party/protobuf/python/google/protobuf/internal/generator_test.py b/third_party/protobuf/python/google/protobuf/internal/generator_test.py index b3f7d9b1..8343aba 100755 --- a/third_party/protobuf/python/google/protobuf/internal/generator_test.py +++ b/third_party/protobuf/python/google/protobuf/internal/generator_test.py @@ -42,8 +42,10 @@ further ensures that we can use Python protocol message objects as we expect. __author__ = 'robinson@google.com (Will Robinson)' import unittest +from google.protobuf.internal import test_bad_identifiers_pb2 from google.protobuf import unittest_custom_options_pb2 from google.protobuf import unittest_import_pb2 +from google.protobuf import unittest_import_public_pb2 from google.protobuf import unittest_mset_pb2 from google.protobuf import unittest_pb2 from google.protobuf import unittest_no_generic_services_pb2 @@ -239,6 +241,29 @@ class GeneratorTest(unittest.TestCase): unittest_pb2._TESTALLTYPES_NESTEDMESSAGE.name in file_type.message_types_by_name) + def testPublicImports(self): + # Test public imports as embedded message. + all_type_proto = unittest_pb2.TestAllTypes() + self.assertEqual(0, all_type_proto.optional_public_import_message.e) + + # PublicImportMessage is actually defined in unittest_import_public_pb2 + # module, and is public imported by unittest_import_pb2 module. + public_import_proto = unittest_import_pb2.PublicImportMessage() + self.assertEqual(0, public_import_proto.e) + self.assertTrue(unittest_import_public_pb2.PublicImportMessage is + unittest_import_pb2.PublicImportMessage) + + def testBadIdentifiers(self): + # We're just testing that the code was imported without problems. + message = test_bad_identifiers_pb2.TestBadIdentifiers() + self.assertEqual(message.Extensions[test_bad_identifiers_pb2.message], + "foo") + self.assertEqual(message.Extensions[test_bad_identifiers_pb2.descriptor], + "bar") + self.assertEqual(message.Extensions[test_bad_identifiers_pb2.reflection], + "baz") + self.assertEqual(message.Extensions[test_bad_identifiers_pb2.service], + "qux") if __name__ == '__main__': unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/message_cpp_test.py b/third_party/protobuf/python/google/protobuf/internal/message_cpp_test.py new file mode 100644 index 0000000..0d84b32 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/message_cpp_test.py @@ -0,0 +1,45 @@ +#! /usr/bin/python +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for google.protobuf.internal.message_cpp.""" + +__author__ = 'shahms@google.com (Shahms King)' + +import os +os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'cpp' + +import unittest +from google.protobuf.internal.message_test import * + + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/message_factory_test.py b/third_party/protobuf/python/google/protobuf/internal/message_factory_test.py new file mode 100644 index 0000000..0bc9be99 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/message_factory_test.py @@ -0,0 +1,113 @@ +#! /usr/bin/python +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Tests for google.protobuf.message_factory.""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + +import unittest +from google.protobuf import descriptor_pb2 +from google.protobuf.internal import factory_test1_pb2 +from google.protobuf.internal import factory_test2_pb2 +from google.protobuf import descriptor_database +from google.protobuf import descriptor_pool +from google.protobuf import message_factory + + +class MessageFactoryTest(unittest.TestCase): + + def setUp(self): + self.factory_test1_fd = descriptor_pb2.FileDescriptorProto.FromString( + factory_test1_pb2.DESCRIPTOR.serialized_pb) + self.factory_test2_fd = descriptor_pb2.FileDescriptorProto.FromString( + factory_test2_pb2.DESCRIPTOR.serialized_pb) + + def _ExerciseDynamicClass(self, cls): + msg = cls() + msg.mandatory = 42 + msg.nested_factory_2_enum = 0 + msg.nested_factory_2_message.value = 'nested message value' + msg.factory_1_message.factory_1_enum = 1 + msg.factory_1_message.nested_factory_1_enum = 0 + msg.factory_1_message.nested_factory_1_message.value = ( + 'nested message value') + msg.factory_1_message.scalar_value = 22 + msg.factory_1_message.list_value.extend(['one', 'two', 'three']) + msg.factory_1_message.list_value.append('four') + msg.factory_1_enum = 1 + msg.nested_factory_1_enum = 0 + msg.nested_factory_1_message.value = 'nested message value' + msg.circular_message.mandatory = 1 + msg.circular_message.circular_message.mandatory = 2 + msg.circular_message.scalar_value = 'one deep' + msg.scalar_value = 'zero deep' + msg.list_value.extend(['four', 'three', 'two']) + msg.list_value.append('one') + msg.grouped.add() + msg.grouped[0].part_1 = 'hello' + msg.grouped[0].part_2 = 'world' + msg.grouped.add(part_1='testing', part_2='123') + msg.loop.loop.mandatory = 2 + msg.loop.loop.loop.loop.mandatory = 4 + serialized = msg.SerializeToString() + converted = factory_test2_pb2.Factory2Message.FromString(serialized) + reserialized = converted.SerializeToString() + self.assertEquals(serialized, reserialized) + result = cls.FromString(reserialized) + self.assertEquals(msg, result) + + def testGetPrototype(self): + db = descriptor_database.DescriptorDatabase() + pool = descriptor_pool.DescriptorPool(db) + db.Add(self.factory_test1_fd) + db.Add(self.factory_test2_fd) + factory = message_factory.MessageFactory() + cls = factory.GetPrototype(pool.FindMessageTypeByName( + 'net.proto2.python.internal.Factory2Message')) + self.assertIsNot(cls, factory_test2_pb2.Factory2Message) + self._ExerciseDynamicClass(cls) + cls2 = factory.GetPrototype(pool.FindMessageTypeByName( + 'net.proto2.python.internal.Factory2Message')) + self.assertIs(cls, cls2) + + def testGetMessages(self): + messages = message_factory.GetMessages([self.factory_test2_fd, + self.factory_test1_fd]) + self.assertContainsSubset( + ['net.proto2.python.internal.Factory2Message', + 'net.proto2.python.internal.Factory1Message'], + messages.keys()) + self._ExerciseDynamicClass( + messages['net.proto2.python.internal.Factory2Message']) + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/message_test.py b/third_party/protobuf/python/google/protobuf/internal/message_test.py index 6517437..53e9d50 100755 --- a/third_party/protobuf/python/google/protobuf/internal/message_test.py +++ b/third_party/protobuf/python/google/protobuf/internal/message_test.py @@ -45,10 +45,15 @@ __author__ = 'gps@google.com (Gregory P. Smith)' import copy import math +import operator +import pickle + import unittest from google.protobuf import unittest_import_pb2 from google.protobuf import unittest_pb2 +from google.protobuf.internal import api_implementation from google.protobuf.internal import test_util +from google.protobuf import message # Python pre-2.6 does not have isinf() or isnan() functions, so we have # to provide our own. @@ -70,9 +75,9 @@ class MessageTest(unittest.TestCase): golden_message = unittest_pb2.TestAllTypes() golden_message.ParseFromString(golden_data) test_util.ExpectAllFieldsSet(self, golden_message) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) golden_copy = copy.deepcopy(golden_message) - self.assertTrue(golden_copy.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_copy.SerializeToString()) def testGoldenExtensions(self): golden_data = test_util.GoldenFile('golden_message').read() @@ -81,9 +86,9 @@ class MessageTest(unittest.TestCase): all_set = unittest_pb2.TestAllExtensions() test_util.SetAllExtensions(all_set) self.assertEquals(all_set, golden_message) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) golden_copy = copy.deepcopy(golden_message) - self.assertTrue(golden_copy.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_copy.SerializeToString()) def testGoldenPackedMessage(self): golden_data = test_util.GoldenFile('golden_packed_fields_message').read() @@ -92,9 +97,9 @@ class MessageTest(unittest.TestCase): all_set = unittest_pb2.TestPackedTypes() test_util.SetAllPackedFields(all_set) self.assertEquals(all_set, golden_message) - self.assertTrue(all_set.SerializeToString() == golden_data) + self.assertEqual(golden_data, all_set.SerializeToString()) golden_copy = copy.deepcopy(golden_message) - self.assertTrue(golden_copy.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_copy.SerializeToString()) def testGoldenPackedExtensions(self): golden_data = test_util.GoldenFile('golden_packed_fields_message').read() @@ -103,9 +108,28 @@ class MessageTest(unittest.TestCase): all_set = unittest_pb2.TestPackedExtensions() test_util.SetAllPackedExtensions(all_set) self.assertEquals(all_set, golden_message) - self.assertTrue(all_set.SerializeToString() == golden_data) + self.assertEqual(golden_data, all_set.SerializeToString()) golden_copy = copy.deepcopy(golden_message) - self.assertTrue(golden_copy.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_copy.SerializeToString()) + + def testPickleSupport(self): + golden_data = test_util.GoldenFile('golden_message').read() + golden_message = unittest_pb2.TestAllTypes() + golden_message.ParseFromString(golden_data) + pickled_message = pickle.dumps(golden_message) + + unpickled_message = pickle.loads(pickled_message) + self.assertEquals(unpickled_message, golden_message) + + def testPickleIncompleteProto(self): + golden_message = unittest_pb2.TestRequired(a=1) + pickled_message = pickle.dumps(golden_message) + + unpickled_message = pickle.loads(pickled_message) + self.assertEquals(unpickled_message, golden_message) + self.assertEquals(unpickled_message.a, 1) + # This is still an incomplete proto - so serializing should fail + self.assertRaises(message.EncodeError, unpickled_message.SerializeToString) def testPositiveInfinity(self): golden_data = ('\x5D\x00\x00\x80\x7F' @@ -118,7 +142,7 @@ class MessageTest(unittest.TestCase): self.assertTrue(IsPosInf(golden_message.optional_double)) self.assertTrue(IsPosInf(golden_message.repeated_float[0])) self.assertTrue(IsPosInf(golden_message.repeated_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) def testNegativeInfinity(self): golden_data = ('\x5D\x00\x00\x80\xFF' @@ -131,7 +155,7 @@ class MessageTest(unittest.TestCase): self.assertTrue(IsNegInf(golden_message.optional_double)) self.assertTrue(IsNegInf(golden_message.repeated_float[0])) self.assertTrue(IsNegInf(golden_message.repeated_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) def testNotANumber(self): golden_data = ('\x5D\x00\x00\xC0\x7F' @@ -144,7 +168,18 @@ class MessageTest(unittest.TestCase): self.assertTrue(isnan(golden_message.optional_double)) self.assertTrue(isnan(golden_message.repeated_float[0])) self.assertTrue(isnan(golden_message.repeated_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + + # The protocol buffer may serialize to any one of multiple different + # representations of a NaN. Rather than verify a specific representation, + # verify the serialized string can be converted into a correctly + # behaving protocol buffer. + serialized = golden_message.SerializeToString() + message = unittest_pb2.TestAllTypes() + message.ParseFromString(serialized) + self.assertTrue(isnan(message.optional_float)) + self.assertTrue(isnan(message.optional_double)) + self.assertTrue(isnan(message.repeated_float[0])) + self.assertTrue(isnan(message.repeated_double[0])) def testPositiveInfinityPacked(self): golden_data = ('\xA2\x06\x04\x00\x00\x80\x7F' @@ -153,7 +188,7 @@ class MessageTest(unittest.TestCase): golden_message.ParseFromString(golden_data) self.assertTrue(IsPosInf(golden_message.packed_float[0])) self.assertTrue(IsPosInf(golden_message.packed_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) def testNegativeInfinityPacked(self): golden_data = ('\xA2\x06\x04\x00\x00\x80\xFF' @@ -162,7 +197,7 @@ class MessageTest(unittest.TestCase): golden_message.ParseFromString(golden_data) self.assertTrue(IsNegInf(golden_message.packed_float[0])) self.assertTrue(IsNegInf(golden_message.packed_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + self.assertEqual(golden_data, golden_message.SerializeToString()) def testNotANumberPacked(self): golden_data = ('\xA2\x06\x04\x00\x00\xC0\x7F' @@ -171,7 +206,12 @@ class MessageTest(unittest.TestCase): golden_message.ParseFromString(golden_data) self.assertTrue(isnan(golden_message.packed_float[0])) self.assertTrue(isnan(golden_message.packed_double[0])) - self.assertTrue(golden_message.SerializeToString() == golden_data) + + serialized = golden_message.SerializeToString() + message = unittest_pb2.TestPackedTypes() + message.ParseFromString(serialized) + self.assertTrue(isnan(message.packed_float[0])) + self.assertTrue(isnan(message.packed_double[0])) def testExtremeFloatValues(self): message = unittest_pb2.TestAllTypes() @@ -218,7 +258,7 @@ class MessageTest(unittest.TestCase): message.ParseFromString(message.SerializeToString()) self.assertTrue(message.optional_float == -kMostNegExponentOneSigBit) - def testExtremeFloatValues(self): + def testExtremeDoubleValues(self): message = unittest_pb2.TestAllTypes() # Most positive exponent, no significand bits set. @@ -338,6 +378,117 @@ class MessageTest(unittest.TestCase): self.assertEqual(message.repeated_nested_message[4].bb, 5) self.assertEqual(message.repeated_nested_message[5].bb, 6) + def testRepeatedCompositeFieldSortArguments(self): + """Check sorting a repeated composite field using list.sort() arguments.""" + message = unittest_pb2.TestAllTypes() + + get_bb = operator.attrgetter('bb') + cmp_bb = lambda a, b: cmp(a.bb, b.bb) + message.repeated_nested_message.add().bb = 1 + message.repeated_nested_message.add().bb = 3 + message.repeated_nested_message.add().bb = 2 + message.repeated_nested_message.add().bb = 6 + message.repeated_nested_message.add().bb = 5 + message.repeated_nested_message.add().bb = 4 + message.repeated_nested_message.sort(key=get_bb) + self.assertEqual([k.bb for k in message.repeated_nested_message], + [1, 2, 3, 4, 5, 6]) + message.repeated_nested_message.sort(key=get_bb, reverse=True) + self.assertEqual([k.bb for k in message.repeated_nested_message], + [6, 5, 4, 3, 2, 1]) + message.repeated_nested_message.sort(sort_function=cmp_bb) + self.assertEqual([k.bb for k in message.repeated_nested_message], + [1, 2, 3, 4, 5, 6]) + message.repeated_nested_message.sort(cmp=cmp_bb, reverse=True) + self.assertEqual([k.bb for k in message.repeated_nested_message], + [6, 5, 4, 3, 2, 1]) + + def testRepeatedScalarFieldSortArguments(self): + """Check sorting a scalar field using list.sort() arguments.""" + message = unittest_pb2.TestAllTypes() + + abs_cmp = lambda a, b: cmp(abs(a), abs(b)) + message.repeated_int32.append(-3) + message.repeated_int32.append(-2) + message.repeated_int32.append(-1) + message.repeated_int32.sort(key=abs) + self.assertEqual(list(message.repeated_int32), [-1, -2, -3]) + message.repeated_int32.sort(key=abs, reverse=True) + self.assertEqual(list(message.repeated_int32), [-3, -2, -1]) + message.repeated_int32.sort(sort_function=abs_cmp) + self.assertEqual(list(message.repeated_int32), [-1, -2, -3]) + message.repeated_int32.sort(cmp=abs_cmp, reverse=True) + self.assertEqual(list(message.repeated_int32), [-3, -2, -1]) + + len_cmp = lambda a, b: cmp(len(a), len(b)) + message.repeated_string.append('aaa') + message.repeated_string.append('bb') + message.repeated_string.append('c') + message.repeated_string.sort(key=len) + self.assertEqual(list(message.repeated_string), ['c', 'bb', 'aaa']) + message.repeated_string.sort(key=len, reverse=True) + self.assertEqual(list(message.repeated_string), ['aaa', 'bb', 'c']) + message.repeated_string.sort(sort_function=len_cmp) + self.assertEqual(list(message.repeated_string), ['c', 'bb', 'aaa']) + message.repeated_string.sort(cmp=len_cmp, reverse=True) + self.assertEqual(list(message.repeated_string), ['aaa', 'bb', 'c']) + + def testParsingMerge(self): + """Check the merge behavior when a required or optional field appears + multiple times in the input.""" + messages = [ + unittest_pb2.TestAllTypes(), + unittest_pb2.TestAllTypes(), + unittest_pb2.TestAllTypes() ] + messages[0].optional_int32 = 1 + messages[1].optional_int64 = 2 + messages[2].optional_int32 = 3 + messages[2].optional_string = 'hello' + + merged_message = unittest_pb2.TestAllTypes() + merged_message.optional_int32 = 3 + merged_message.optional_int64 = 2 + merged_message.optional_string = 'hello' + + generator = unittest_pb2.TestParsingMerge.RepeatedFieldsGenerator() + generator.field1.extend(messages) + generator.field2.extend(messages) + generator.field3.extend(messages) + generator.ext1.extend(messages) + generator.ext2.extend(messages) + generator.group1.add().field1.MergeFrom(messages[0]) + generator.group1.add().field1.MergeFrom(messages[1]) + generator.group1.add().field1.MergeFrom(messages[2]) + generator.group2.add().field1.MergeFrom(messages[0]) + generator.group2.add().field1.MergeFrom(messages[1]) + generator.group2.add().field1.MergeFrom(messages[2]) + + data = generator.SerializeToString() + parsing_merge = unittest_pb2.TestParsingMerge() + parsing_merge.ParseFromString(data) + + # Required and optional fields should be merged. + self.assertEqual(parsing_merge.required_all_types, merged_message) + self.assertEqual(parsing_merge.optional_all_types, merged_message) + self.assertEqual(parsing_merge.optionalgroup.optional_group_all_types, + merged_message) + self.assertEqual(parsing_merge.Extensions[ + unittest_pb2.TestParsingMerge.optional_ext], + merged_message) + + # Repeated fields should not be merged. + self.assertEqual(len(parsing_merge.repeated_all_types), 3) + self.assertEqual(len(parsing_merge.repeatedgroup), 3) + self.assertEqual(len(parsing_merge.Extensions[ + unittest_pb2.TestParsingMerge.repeated_ext]), 3) + + + def testSortEmptyRepeatedCompositeContainer(self): + """Exercise a scenario that has led to segfaults in the past. + """ + m = unittest_pb2.TestAllTypes() + m.repeated_nested_message.sort() + if __name__ == '__main__': unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/more_extensions_dynamic.proto b/third_party/protobuf/python/google/protobuf/internal/more_extensions_dynamic.proto new file mode 100644 index 0000000..df98ac4 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/more_extensions_dynamic.proto @@ -0,0 +1,49 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: jasonh@google.com (Jason Hsueh) +// +// This file is used to test a corner case in the CPP implementation where the +// generated C++ type is available for the extendee, but the extension is +// defined in a file whose C++ type is not in the binary. + + +import "google/protobuf/internal/more_extensions.proto"; + +package google.protobuf.internal; + +message DynamicMessageType { + optional int32 a = 1; +} + +extend ExtendedMessage { + optional int32 dynamic_int32_extension = 100; + optional DynamicMessageType dynamic_message_extension = 101; +} diff --git a/third_party/protobuf/python/google/protobuf/internal/python_message.py b/third_party/protobuf/python/google/protobuf/internal/python_message.py index 66fca91..4bea57a 100644 --- a/third_party/protobuf/python/google/protobuf/internal/python_message.py +++ b/third_party/protobuf/python/google/protobuf/internal/python_message.py @@ -54,6 +54,7 @@ try: from cStringIO import StringIO except ImportError: from StringIO import StringIO +import copy_reg import struct import weakref @@ -61,6 +62,7 @@ import weakref from google.protobuf.internal import containers from google.protobuf.internal import decoder from google.protobuf.internal import encoder +from google.protobuf.internal import enum_type_wrapper from google.protobuf.internal import message_listener as message_listener_mod from google.protobuf.internal import type_checkers from google.protobuf.internal import wire_format @@ -71,9 +73,10 @@ from google.protobuf import text_format _FieldDescriptor = descriptor_mod.FieldDescriptor -def NewMessage(descriptor, dictionary): +def NewMessage(bases, descriptor, dictionary): _AddClassAttributesForNestedExtensions(descriptor, dictionary) _AddSlots(descriptor, dictionary) + return bases def InitMessage(descriptor, cls): @@ -96,6 +99,7 @@ def InitMessage(descriptor, cls): _AddStaticMethods(cls) _AddMessageMethods(descriptor, cls) _AddPrivateHelperMethods(cls) + copy_reg.pickle(cls, lambda obj: (cls, (), obj.__getstate__())) # Stateless helpers for GeneratedProtocolMessageType below. @@ -145,6 +149,10 @@ def _VerifyExtensionHandle(message, extension_handle): if not extension_handle.is_extension: raise KeyError('"%s" is not an extension.' % extension_handle.full_name) + if not extension_handle.containing_type: + raise KeyError('"%s" is missing a containing_type.' + % extension_handle.full_name) + if extension_handle.containing_type is not message.DESCRIPTOR: raise KeyError('Extension "%s" extends message type "%s", but this ' 'message is of type "%s".' % @@ -164,6 +172,7 @@ def _AddSlots(message_descriptor, dictionary): dictionary['__slots__'] = ['_cached_byte_size', '_cached_byte_size_dirty', '_fields', + '_unknown_fields', '_is_present_in_parent', '_listener', '_listener_for_children', @@ -224,11 +233,14 @@ def _AddClassAttributesForNestedExtensions(descriptor, dictionary): def _AddEnumValues(descriptor, cls): """Sets class-level attributes for all enum fields defined in this message. + Also exporting a class-level object that can name enum values. + Args: descriptor: Descriptor object for this message type. cls: Class we're constructing for this message type. """ for enum_type in descriptor.enum_types: + setattr(cls, enum_type.name, enum_type_wrapper.EnumTypeWrapper(enum_type)) for enum_value in enum_type.values: setattr(cls, enum_value.name, enum_value.number) @@ -248,7 +260,7 @@ def _DefaultValueConstructorForField(field): """ if field.label == _FieldDescriptor.LABEL_REPEATED: - if field.default_value != []: + if field.has_default_value and field.default_value != []: raise ValueError('Repeated field default value not empty list: %s' % ( field.default_value)) if field.cpp_type == _FieldDescriptor.CPPTYPE_MESSAGE: @@ -276,6 +288,8 @@ def _DefaultValueConstructorForField(field): return MakeSubMessageDefault def MakeScalarDefault(message): + # TODO(protobuf-team): This may be broken since there may not be + # default_value. Combine with has_default_value somehow. return field.default_value return MakeScalarDefault @@ -287,6 +301,9 @@ def _AddInitMethod(message_descriptor, cls): self._cached_byte_size = 0 self._cached_byte_size_dirty = len(kwargs) > 0 self._fields = {} + # _unknown_fields is () when empty for efficiency, and will be turned into + # a list if fields are added. + self._unknown_fields = () self._is_present_in_parent = False self._listener = message_listener_mod.NullMessageListener() self._listener_for_children = _Listener(self) @@ -428,6 +445,8 @@ def _AddPropertiesForNonRepeatedScalarField(field, cls): valid_values = set() def getter(self): + # TODO(protobuf-team): This may be broken since there may not be + # default_value. Combine with has_default_value somehow. return self._fields.get(field, default_value) getter.__module__ = None getter.__doc__ = 'Getter for %s.' % proto_field_name @@ -462,13 +481,18 @@ def _AddPropertiesForNonRepeatedCompositeField(field, cls): # for non-repeated scalars. proto_field_name = field.name property_name = _PropertyName(proto_field_name) + + # TODO(komarek): Can anyone explain to me why we cache the message_type this + # way, instead of referring to field.message_type inside of getter(self)? + # What if someone sets message_type later on (which makes for simpler + # dyanmic proto descriptor and class creation code). message_type = field.message_type def getter(self): field_value = self._fields.get(field) if field_value is None: # Construct a new object to represent this field. - field_value = message_type._concrete_class() + field_value = message_type._concrete_class() # use field.message_type? field_value._SetListener(self._listener_for_children) # Atomically check if another thread has preempted us and, if not, swap @@ -620,6 +644,7 @@ def _AddClearMethod(message_descriptor, cls): def Clear(self): # Clear fields. self._fields = {} + self._unknown_fields = () self._Modified() cls.Clear = Clear @@ -649,7 +674,16 @@ def _AddEqualsMethod(message_descriptor, cls): if self is other: return True - return self.ListFields() == other.ListFields() + if not self.ListFields() == other.ListFields(): + return False + + # Sort unknown fields because their order shouldn't affect equality test. + unknown_fields = list(self._unknown_fields) + unknown_fields.sort() + other_unknown_fields = list(other._unknown_fields) + other_unknown_fields.sort() + + return unknown_fields == other_unknown_fields cls.__eq__ = __eq__ @@ -710,6 +744,9 @@ def _AddByteSizeMethod(message_descriptor, cls): for field_descriptor, field_value in self.ListFields(): size += field_descriptor._sizer(field_value) + for tag_bytes, value_bytes in self._unknown_fields: + size += len(tag_bytes) + len(value_bytes) + self._cached_byte_size = size self._cached_byte_size_dirty = False self._listener_for_children.dirty = False @@ -726,8 +763,8 @@ def _AddSerializeToStringMethod(message_descriptor, cls): errors = [] if not self.IsInitialized(): raise message_mod.EncodeError( - 'Message is missing required fields: ' + - ','.join(self.FindInitializationErrors())) + 'Message %s is missing required fields: %s' % ( + self.DESCRIPTOR.full_name, ','.join(self.FindInitializationErrors()))) return self.SerializePartialToString() cls.SerializeToString = SerializeToString @@ -744,6 +781,9 @@ def _AddSerializePartialToStringMethod(message_descriptor, cls): def InternalSerialize(self, write_bytes): for field_descriptor, field_value in self.ListFields(): field_descriptor._encoder(write_bytes, field_value) + for tag_bytes, value_bytes in self._unknown_fields: + write_bytes(tag_bytes) + write_bytes(value_bytes) cls._InternalSerialize = InternalSerialize @@ -770,13 +810,18 @@ def _AddMergeFromStringMethod(message_descriptor, cls): def InternalParse(self, buffer, pos, end): self._Modified() field_dict = self._fields + unknown_field_list = self._unknown_fields while pos != end: (tag_bytes, new_pos) = local_ReadTag(buffer, pos) field_decoder = decoders_by_tag.get(tag_bytes) if field_decoder is None: + value_start_pos = new_pos new_pos = local_SkipField(buffer, new_pos, end, tag_bytes) if new_pos == -1: return pos + if not unknown_field_list: + unknown_field_list = self._unknown_fields = [] + unknown_field_list.append((tag_bytes, buffer[value_start_pos:new_pos])) pos = new_pos else: pos = field_decoder(buffer, new_pos, end, self, field_dict) @@ -873,7 +918,8 @@ def _AddMergeFromMethod(cls): def MergeFrom(self, msg): if not isinstance(msg, cls): raise TypeError( - "Parameter to MergeFrom() must be instance of same class.") + "Parameter to MergeFrom() must be instance of same class: " + "expected %s got %s." % (cls.__name__, type(msg).__name__)) assert msg is not self self._Modified() @@ -898,6 +944,12 @@ def _AddMergeFromMethod(cls): field_value.MergeFrom(value) else: self._fields[field] = value + + if msg._unknown_fields: + if not self._unknown_fields: + self._unknown_fields = [] + self._unknown_fields.extend(msg._unknown_fields) + cls.MergeFrom = MergeFrom diff --git a/third_party/protobuf/python/google/protobuf/internal/reflection_cpp_generated_test.py b/third_party/protobuf/python/google/protobuf/internal/reflection_cpp_generated_test.py new file mode 100644 index 0000000..2a0a512 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/reflection_cpp_generated_test.py @@ -0,0 +1,91 @@ +#! /usr/bin/python +# -*- coding: utf-8 -*- +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Unittest for reflection.py, which tests the generated C++ implementation.""" + +__author__ = 'jasonh@google.com (Jason Hsueh)' + +import os +os.environ['PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION'] = 'cpp' + +import unittest +from google.protobuf.internal import api_implementation +from google.protobuf.internal import more_extensions_dynamic_pb2 +from google.protobuf.internal import more_extensions_pb2 +from google.protobuf.internal.reflection_test import * + + +class ReflectionCppTest(unittest.TestCase): + def testImplementationSetting(self): + self.assertEqual('cpp', api_implementation.Type()) + + def testExtensionOfGeneratedTypeInDynamicFile(self): + """Tests that a file built dynamically can extend a generated C++ type. + + The C++ implementation uses a DescriptorPool that has the generated + DescriptorPool as an underlay. Typically, a type can only find + extensions in its own pool. With the python C-extension, the generated C++ + extendee may be available, but not the extension. This tests that the + C-extension implements the correct special handling to make such extensions + available. + """ + pb1 = more_extensions_pb2.ExtendedMessage() + # Test that basic accessors work. + self.assertFalse( + pb1.HasExtension(more_extensions_dynamic_pb2.dynamic_int32_extension)) + self.assertFalse( + pb1.HasExtension(more_extensions_dynamic_pb2.dynamic_message_extension)) + pb1.Extensions[more_extensions_dynamic_pb2.dynamic_int32_extension] = 17 + pb1.Extensions[more_extensions_dynamic_pb2.dynamic_message_extension].a = 24 + self.assertTrue( + pb1.HasExtension(more_extensions_dynamic_pb2.dynamic_int32_extension)) + self.assertTrue( + pb1.HasExtension(more_extensions_dynamic_pb2.dynamic_message_extension)) + + # Now serialize the data and parse to a new message. + pb2 = more_extensions_pb2.ExtendedMessage() + pb2.MergeFromString(pb1.SerializeToString()) + + self.assertTrue( + pb2.HasExtension(more_extensions_dynamic_pb2.dynamic_int32_extension)) + self.assertTrue( + pb2.HasExtension(more_extensions_dynamic_pb2.dynamic_message_extension)) + self.assertEqual( + 17, pb2.Extensions[more_extensions_dynamic_pb2.dynamic_int32_extension]) + self.assertEqual( + 24, + pb2.Extensions[more_extensions_dynamic_pb2.dynamic_message_extension].a) + + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/reflection_test.py b/third_party/protobuf/python/google/protobuf/internal/reflection_test.py index 7b9d339..ed28646 100755 --- a/third_party/protobuf/python/google/protobuf/internal/reflection_test.py +++ b/third_party/protobuf/python/google/protobuf/internal/reflection_test.py @@ -37,6 +37,7 @@ pure-Python protocol compiler. __author__ = 'robinson@google.com (Will Robinson)' +import gc import operator import struct @@ -318,15 +319,6 @@ class ReflectionTest(unittest.TestCase): # ...and ensure that the scalar field has returned to its default. self.assertEqual(0, getattr(composite_field, scalar_field_name)) - # Finally, ensure that modifications to the old composite field object - # don't have any effect on the parent. Possible only with the pure-python - # implementation of the API. - # - # (NOTE that when we clear the composite field in the parent, we actually - # don't recursively clear down the tree. Instead, we just disconnect the - # cleared composite from the tree.) - if api_implementation.Type() != 'python': - return self.assertTrue(old_composite_field is not composite_field) setattr(old_composite_field, scalar_field_name, new_val) self.assertTrue(not composite_field.HasField(scalar_field_name)) @@ -348,8 +340,6 @@ class ReflectionTest(unittest.TestCase): nested.bb = 23 def testDisconnectingNestedMessageBeforeSettingField(self): - if api_implementation.Type() != 'python': - return proto = unittest_pb2.TestAllTypes() nested = proto.optional_nested_message proto.ClearField('optional_nested_message') # Should disconnect from parent @@ -358,6 +348,64 @@ class ReflectionTest(unittest.TestCase): self.assertTrue(not proto.HasField('optional_nested_message')) self.assertEqual(0, proto.optional_nested_message.bb) + def testGetDefaultMessageAfterDisconnectingDefaultMessage(self): + proto = unittest_pb2.TestAllTypes() + nested = proto.optional_nested_message + proto.ClearField('optional_nested_message') + del proto + del nested + # Force a garbage collect so that the underlying CMessages are freed along + # with the Messages they point to. This is to make sure we're not deleting + # default message instances. + gc.collect() + proto = unittest_pb2.TestAllTypes() + nested = proto.optional_nested_message + + def testDisconnectingNestedMessageAfterSettingField(self): + proto = unittest_pb2.TestAllTypes() + nested = proto.optional_nested_message + nested.bb = 5 + self.assertTrue(proto.HasField('optional_nested_message')) + proto.ClearField('optional_nested_message') # Should disconnect from parent + self.assertEqual(5, nested.bb) + self.assertEqual(0, proto.optional_nested_message.bb) + self.assertTrue(nested is not proto.optional_nested_message) + nested.bb = 23 + self.assertTrue(not proto.HasField('optional_nested_message')) + self.assertEqual(0, proto.optional_nested_message.bb) + + def testDisconnectingNestedMessageBeforeGettingField(self): + proto = unittest_pb2.TestAllTypes() + self.assertTrue(not proto.HasField('optional_nested_message')) + proto.ClearField('optional_nested_message') + self.assertTrue(not proto.HasField('optional_nested_message')) + + def testDisconnectingNestedMessageAfterMerge(self): + # This test exercises the code path that does not use ReleaseMessage(). + # The underlying fear is that if we use ReleaseMessage() incorrectly, + # we will have memory leaks. It's hard to check that that doesn't happen, + # but at least we can exercise that code path to make sure it works. + proto1 = unittest_pb2.TestAllTypes() + proto2 = unittest_pb2.TestAllTypes() + proto2.optional_nested_message.bb = 5 + proto1.MergeFrom(proto2) + self.assertTrue(proto1.HasField('optional_nested_message')) + proto1.ClearField('optional_nested_message') + self.assertTrue(not proto1.HasField('optional_nested_message')) + + def testDisconnectingLazyNestedMessage(self): + # This test exercises releasing a nested message that is lazy. This test + # only exercises real code in the C++ implementation as Python does not + # support lazy parsing, but the current C++ implementation results in + # memory corruption and a crash. + if api_implementation.Type() != 'python': + return + proto = unittest_pb2.TestAllTypes() + proto.optional_lazy_message.bb = 5 + proto.ClearField('optional_lazy_message') + del proto + gc.collect() + def testHasBitsWhenModifyingRepeatedFields(self): # Test nesting when we add an element to a repeated field in a submessage. proto = unittest_pb2.TestNestedMessageHasBits() @@ -635,6 +683,77 @@ class ReflectionTest(unittest.TestCase): self.assertEqual(3, proto.BAZ) self.assertEqual(3, unittest_pb2.TestAllTypes.BAZ) + def testEnum_Name(self): + self.assertEqual('FOREIGN_FOO', + unittest_pb2.ForeignEnum.Name(unittest_pb2.FOREIGN_FOO)) + self.assertEqual('FOREIGN_BAR', + unittest_pb2.ForeignEnum.Name(unittest_pb2.FOREIGN_BAR)) + self.assertEqual('FOREIGN_BAZ', + unittest_pb2.ForeignEnum.Name(unittest_pb2.FOREIGN_BAZ)) + self.assertRaises(ValueError, + unittest_pb2.ForeignEnum.Name, 11312) + + proto = unittest_pb2.TestAllTypes() + self.assertEqual('FOO', + proto.NestedEnum.Name(proto.FOO)) + self.assertEqual('FOO', + unittest_pb2.TestAllTypes.NestedEnum.Name(proto.FOO)) + self.assertEqual('BAR', + proto.NestedEnum.Name(proto.BAR)) + self.assertEqual('BAR', + unittest_pb2.TestAllTypes.NestedEnum.Name(proto.BAR)) + self.assertEqual('BAZ', + proto.NestedEnum.Name(proto.BAZ)) + self.assertEqual('BAZ', + unittest_pb2.TestAllTypes.NestedEnum.Name(proto.BAZ)) + self.assertRaises(ValueError, + proto.NestedEnum.Name, 11312) + self.assertRaises(ValueError, + unittest_pb2.TestAllTypes.NestedEnum.Name, 11312) + + def testEnum_Value(self): + self.assertEqual(unittest_pb2.FOREIGN_FOO, + unittest_pb2.ForeignEnum.Value('FOREIGN_FOO')) + self.assertEqual(unittest_pb2.FOREIGN_BAR, + unittest_pb2.ForeignEnum.Value('FOREIGN_BAR')) + self.assertEqual(unittest_pb2.FOREIGN_BAZ, + unittest_pb2.ForeignEnum.Value('FOREIGN_BAZ')) + self.assertRaises(ValueError, + unittest_pb2.ForeignEnum.Value, 'FO') + + proto = unittest_pb2.TestAllTypes() + self.assertEqual(proto.FOO, + proto.NestedEnum.Value('FOO')) + self.assertEqual(proto.FOO, + unittest_pb2.TestAllTypes.NestedEnum.Value('FOO')) + self.assertEqual(proto.BAR, + proto.NestedEnum.Value('BAR')) + self.assertEqual(proto.BAR, + unittest_pb2.TestAllTypes.NestedEnum.Value('BAR')) + self.assertEqual(proto.BAZ, + proto.NestedEnum.Value('BAZ')) + self.assertEqual(proto.BAZ, + unittest_pb2.TestAllTypes.NestedEnum.Value('BAZ')) + self.assertRaises(ValueError, + proto.NestedEnum.Value, 'Foo') + self.assertRaises(ValueError, + unittest_pb2.TestAllTypes.NestedEnum.Value, 'Foo') + + def testEnum_KeysAndValues(self): + self.assertEqual(['FOREIGN_FOO', 'FOREIGN_BAR', 'FOREIGN_BAZ'], + unittest_pb2.ForeignEnum.keys()) + self.assertEqual([4, 5, 6], + unittest_pb2.ForeignEnum.values()) + self.assertEqual([('FOREIGN_FOO', 4), ('FOREIGN_BAR', 5), + ('FOREIGN_BAZ', 6)], + unittest_pb2.ForeignEnum.items()) + + proto = unittest_pb2.TestAllTypes() + self.assertEqual(['FOO', 'BAR', 'BAZ'], proto.NestedEnum.keys()) + self.assertEqual([1, 2, 3], proto.NestedEnum.values()) + self.assertEqual([('FOO', 1), ('BAR', 2), ('BAZ', 3)], + proto.NestedEnum.items()) + def testRepeatedScalars(self): proto = unittest_pb2.TestAllTypes() @@ -826,6 +945,35 @@ class ReflectionTest(unittest.TestCase): self.assertEqual(1, len(proto.repeated_nested_message)) self.assertEqual(23, proto.repeated_nested_message[0].bb) + def testRepeatedCompositeRemove(self): + proto = unittest_pb2.TestAllTypes() + + self.assertEqual(0, len(proto.repeated_nested_message)) + m0 = proto.repeated_nested_message.add() + # Need to set some differentiating variable so m0 != m1 != m2: + m0.bb = len(proto.repeated_nested_message) + m1 = proto.repeated_nested_message.add() + m1.bb = len(proto.repeated_nested_message) + self.assertTrue(m0 != m1) + m2 = proto.repeated_nested_message.add() + m2.bb = len(proto.repeated_nested_message) + self.assertListsEqual([m0, m1, m2], proto.repeated_nested_message) + + self.assertEqual(3, len(proto.repeated_nested_message)) + proto.repeated_nested_message.remove(m0) + self.assertEqual(2, len(proto.repeated_nested_message)) + self.assertEqual(m1, proto.repeated_nested_message[0]) + self.assertEqual(m2, proto.repeated_nested_message[1]) + + # Removing m0 again or removing None should raise error + self.assertRaises(ValueError, proto.repeated_nested_message.remove, m0) + self.assertRaises(ValueError, proto.repeated_nested_message.remove, None) + self.assertEqual(2, len(proto.repeated_nested_message)) + + proto.repeated_nested_message.remove(m2) + self.assertEqual(1, len(proto.repeated_nested_message)) + self.assertEqual(m1, proto.repeated_nested_message[0]) + def testHandWrittenReflection(self): # Hand written extensions are only supported by the pure-Python # implementation of the API. @@ -856,6 +1004,68 @@ class ReflectionTest(unittest.TestCase): self.assertEqual(23, myproto_instance.foo_field) self.assertTrue(myproto_instance.HasField('foo_field')) + def testDescriptorProtoSupport(self): + # Hand written descriptors/reflection are only supported by the pure-Python + # implementation of the API. + if api_implementation.Type() != 'python': + return + + def AddDescriptorField(proto, field_name, field_type): + AddDescriptorField.field_index += 1 + new_field = proto.field.add() + new_field.name = field_name + new_field.type = field_type + new_field.number = AddDescriptorField.field_index + new_field.label = descriptor_pb2.FieldDescriptorProto.LABEL_OPTIONAL + + AddDescriptorField.field_index = 0 + + desc_proto = descriptor_pb2.DescriptorProto() + desc_proto.name = 'Car' + fdp = descriptor_pb2.FieldDescriptorProto + AddDescriptorField(desc_proto, 'name', fdp.TYPE_STRING) + AddDescriptorField(desc_proto, 'year', fdp.TYPE_INT64) + AddDescriptorField(desc_proto, 'automatic', fdp.TYPE_BOOL) + AddDescriptorField(desc_proto, 'price', fdp.TYPE_DOUBLE) + # Add a repeated field + AddDescriptorField.field_index += 1 + new_field = desc_proto.field.add() + new_field.name = 'owners' + new_field.type = fdp.TYPE_STRING + new_field.number = AddDescriptorField.field_index + new_field.label = descriptor_pb2.FieldDescriptorProto.LABEL_REPEATED + + desc = descriptor.MakeDescriptor(desc_proto) + self.assertTrue(desc.fields_by_name.has_key('name')) + self.assertTrue(desc.fields_by_name.has_key('year')) + self.assertTrue(desc.fields_by_name.has_key('automatic')) + self.assertTrue(desc.fields_by_name.has_key('price')) + self.assertTrue(desc.fields_by_name.has_key('owners')) + + class CarMessage(message.Message): + __metaclass__ = reflection.GeneratedProtocolMessageType + DESCRIPTOR = desc + + prius = CarMessage() + prius.name = 'prius' + prius.year = 2010 + prius.automatic = True + prius.price = 25134.75 + prius.owners.extend(['bob', 'susan']) + + serialized_prius = prius.SerializeToString() + new_prius = reflection.ParseMessage(desc, serialized_prius) + self.assertTrue(new_prius is not prius) + self.assertEqual(prius, new_prius) + + # these are unnecessary assuming message equality works as advertised but + # explicitly check to be safe since we're mucking about in metaclass foo + self.assertEqual(prius.name, new_prius.name) + self.assertEqual(prius.year, new_prius.year) + self.assertEqual(prius.automatic, new_prius.automatic) + self.assertEqual(prius.price, new_prius.price) + self.assertEqual(prius.owners, new_prius.owners) + def testTopLevelExtensionsForOptionalScalar(self): extendee_proto = unittest_pb2.TestAllExtensions() extension = unittest_pb2.optional_int32_extension @@ -1243,7 +1453,12 @@ class ReflectionTest(unittest.TestCase): def testClear(self): proto = unittest_pb2.TestAllTypes() - test_util.SetAllFields(proto) + # C++ implementation does not support lazy fields right now so leave it + # out for now. + if api_implementation.Type() == 'python': + test_util.SetAllFields(proto) + else: + test_util.SetAllNonLazyFields(proto) # Clear the message. proto.Clear() self.assertEquals(proto.ByteSize(), 0) @@ -1259,6 +1474,33 @@ class ReflectionTest(unittest.TestCase): empty_proto = unittest_pb2.TestAllExtensions() self.assertEquals(proto, empty_proto) + def testDisconnectingBeforeClear(self): + proto = unittest_pb2.TestAllTypes() + nested = proto.optional_nested_message + proto.Clear() + self.assertTrue(nested is not proto.optional_nested_message) + nested.bb = 23 + self.assertTrue(not proto.HasField('optional_nested_message')) + self.assertEqual(0, proto.optional_nested_message.bb) + + proto = unittest_pb2.TestAllTypes() + nested = proto.optional_nested_message + nested.bb = 5 + foreign = proto.optional_foreign_message + foreign.c = 6 + + proto.Clear() + self.assertTrue(nested is not proto.optional_nested_message) + self.assertTrue(foreign is not proto.optional_foreign_message) + self.assertEqual(5, nested.bb) + self.assertEqual(6, foreign.c) + nested.bb = 15 + foreign.c = 16 + self.assertTrue(not proto.HasField('optional_nested_message')) + self.assertEqual(0, proto.optional_nested_message.bb) + self.assertTrue(not proto.HasField('optional_foreign_message')) + self.assertEqual(0, proto.optional_foreign_message.c) + def assertInitialized(self, proto): self.assertTrue(proto.IsInitialized()) # Neither method should raise an exception. @@ -1408,7 +1650,7 @@ class ReflectionTest(unittest.TestCase): unicode_decode_failed = False try: message2.MergeFromString(bytes) - except UnicodeDecodeError, e: + except UnicodeDecodeError as e: unicode_decode_failed = True string_field = message2.str self.assertTrue(unicode_decode_failed or type(string_field) == str) @@ -2119,7 +2361,7 @@ class SerializationTest(unittest.TestCase): """This method checks if the excpetion type and message are as expected.""" try: callable_obj() - except exc_class, ex: + except exc_class as ex: # Check if the exception message is the right one. self.assertEqual(exception, str(ex)) return @@ -2131,15 +2373,22 @@ class SerializationTest(unittest.TestCase): self._CheckRaises( message.EncodeError, proto.SerializeToString, - 'Message is missing required fields: a,b,c') + 'Message protobuf_unittest.TestRequired is missing required fields: ' + 'a,b,c') # Shouldn't raise exceptions. partial = proto.SerializePartialToString() + proto2 = unittest_pb2.TestRequired() + self.assertFalse(proto2.HasField('a')) + # proto2 ParseFromString does not check that required fields are set. + proto2.ParseFromString(partial) + self.assertFalse(proto2.HasField('a')) + proto.a = 1 self._CheckRaises( message.EncodeError, proto.SerializeToString, - 'Message is missing required fields: b,c') + 'Message protobuf_unittest.TestRequired is missing required fields: b,c') # Shouldn't raise exceptions. partial = proto.SerializePartialToString() @@ -2147,7 +2396,7 @@ class SerializationTest(unittest.TestCase): self._CheckRaises( message.EncodeError, proto.SerializeToString, - 'Message is missing required fields: c') + 'Message protobuf_unittest.TestRequired is missing required fields: c') # Shouldn't raise exceptions. partial = proto.SerializePartialToString() @@ -2176,7 +2425,8 @@ class SerializationTest(unittest.TestCase): self._CheckRaises( message.EncodeError, proto.SerializeToString, - 'Message is missing required fields: ' + 'Message protobuf_unittest.TestRequiredForeign ' + 'is missing required fields: ' 'optional_message.b,optional_message.c') proto.optional_message.b = 2 @@ -2188,7 +2438,7 @@ class SerializationTest(unittest.TestCase): self._CheckRaises( message.EncodeError, proto.SerializeToString, - 'Message is missing required fields: ' + 'Message protobuf_unittest.TestRequiredForeign is missing required fields: ' 'repeated_message[0].b,repeated_message[0].c,' 'repeated_message[1].a,repeated_message[1].c') diff --git a/third_party/protobuf/python/google/protobuf/internal/test_bad_identifiers.proto b/third_party/protobuf/python/google/protobuf/internal/test_bad_identifiers.proto new file mode 100644 index 0000000..6a82299 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/test_bad_identifiers.proto @@ -0,0 +1,52 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: kenton@google.com (Kenton Varda) + + +package protobuf_unittest; + +option py_generic_services = true; + +message TestBadIdentifiers { + extensions 100 to max; +} + +// Make sure these reasonable extension names don't conflict with internal +// variables. +extend TestBadIdentifiers { + optional string message = 100 [default="foo"]; + optional string descriptor = 101 [default="bar"]; + optional string reflection = 102 [default="baz"]; + optional string service = 103 [default="qux"]; +} + +message AnotherMessage {} +service AnotherService {} diff --git a/third_party/protobuf/python/google/protobuf/internal/test_util.py b/third_party/protobuf/python/google/protobuf/internal/test_util.py index 1df1619..be8ae7b 100755 --- a/third_party/protobuf/python/google/protobuf/internal/test_util.py +++ b/third_party/protobuf/python/google/protobuf/internal/test_util.py @@ -42,8 +42,8 @@ from google.protobuf import unittest_import_pb2 from google.protobuf import unittest_pb2 -def SetAllFields(message): - """Sets every field in the message to a unique value. +def SetAllNonLazyFields(message): + """Sets every non-lazy field in the message to a unique value. Args: message: A unittest_pb2.TestAllTypes instance. @@ -79,6 +79,7 @@ def SetAllFields(message): message.optional_nested_message.bb = 118 message.optional_foreign_message.c = 119 message.optional_import_message.d = 120 + message.optional_public_import_message.e = 126 message.optional_nested_enum = unittest_pb2.TestAllTypes.BAZ message.optional_foreign_enum = unittest_pb2.FOREIGN_BAZ @@ -111,6 +112,7 @@ def SetAllFields(message): message.repeated_nested_message.add().bb = 218 message.repeated_foreign_message.add().c = 219 message.repeated_import_message.add().d = 220 + message.repeated_lazy_message.add().bb = 227 message.repeated_nested_enum.append(unittest_pb2.TestAllTypes.BAR) message.repeated_foreign_enum.append(unittest_pb2.FOREIGN_BAR) @@ -140,6 +142,7 @@ def SetAllFields(message): message.repeated_nested_message.add().bb = 318 message.repeated_foreign_message.add().c = 319 message.repeated_import_message.add().d = 320 + message.repeated_lazy_message.add().bb = 327 message.repeated_nested_enum.append(unittest_pb2.TestAllTypes.BAZ) message.repeated_foreign_enum.append(unittest_pb2.FOREIGN_BAZ) @@ -176,6 +179,11 @@ def SetAllFields(message): message.default_cord = '425' +def SetAllFields(message): + SetAllNonLazyFields(message) + message.optional_lazy_message.bb = 127 + + def SetAllExtensions(message): """Sets every extension in the message to a unique value. @@ -211,6 +219,8 @@ def SetAllExtensions(message): extensions[pb2.optional_nested_message_extension].bb = 118 extensions[pb2.optional_foreign_message_extension].c = 119 extensions[pb2.optional_import_message_extension].d = 120 + extensions[pb2.optional_public_import_message_extension].e = 126 + extensions[pb2.optional_lazy_message_extension].bb = 127 extensions[pb2.optional_nested_enum_extension] = pb2.TestAllTypes.BAZ extensions[pb2.optional_nested_enum_extension] = pb2.TestAllTypes.BAZ @@ -244,6 +254,7 @@ def SetAllExtensions(message): extensions[pb2.repeated_nested_message_extension].add().bb = 218 extensions[pb2.repeated_foreign_message_extension].add().c = 219 extensions[pb2.repeated_import_message_extension].add().d = 220 + extensions[pb2.repeated_lazy_message_extension].add().bb = 227 extensions[pb2.repeated_nested_enum_extension].append(pb2.TestAllTypes.BAR) extensions[pb2.repeated_foreign_enum_extension].append(pb2.FOREIGN_BAR) @@ -273,6 +284,7 @@ def SetAllExtensions(message): extensions[pb2.repeated_nested_message_extension].add().bb = 318 extensions[pb2.repeated_foreign_message_extension].add().c = 319 extensions[pb2.repeated_import_message_extension].add().d = 320 + extensions[pb2.repeated_lazy_message_extension].add().bb = 327 extensions[pb2.repeated_nested_enum_extension].append(pb2.TestAllTypes.BAZ) extensions[pb2.repeated_foreign_enum_extension].append(pb2.FOREIGN_BAZ) @@ -407,6 +419,8 @@ def ExpectAllFieldsSet(test_case, message): test_case.assertEqual(118, message.optional_nested_message.bb) test_case.assertEqual(119, message.optional_foreign_message.c) test_case.assertEqual(120, message.optional_import_message.d) + test_case.assertEqual(126, message.optional_public_import_message.e) + test_case.assertEqual(127, message.optional_lazy_message.bb) test_case.assertEqual(unittest_pb2.TestAllTypes.BAZ, message.optional_nested_enum) @@ -464,6 +478,7 @@ def ExpectAllFieldsSet(test_case, message): test_case.assertEqual(218, message.repeated_nested_message[0].bb) test_case.assertEqual(219, message.repeated_foreign_message[0].c) test_case.assertEqual(220, message.repeated_import_message[0].d) + test_case.assertEqual(227, message.repeated_lazy_message[0].bb) test_case.assertEqual(unittest_pb2.TestAllTypes.BAR, message.repeated_nested_enum[0]) @@ -492,6 +507,7 @@ def ExpectAllFieldsSet(test_case, message): test_case.assertEqual(318, message.repeated_nested_message[1].bb) test_case.assertEqual(319, message.repeated_foreign_message[1].c) test_case.assertEqual(320, message.repeated_import_message[1].d) + test_case.assertEqual(327, message.repeated_lazy_message[1].bb) test_case.assertEqual(unittest_pb2.TestAllTypes.BAZ, message.repeated_nested_enum[1]) diff --git a/third_party/protobuf/python/google/protobuf/internal/text_format_test.py b/third_party/protobuf/python/google/protobuf/internal/text_format_test.py index 73d97d1..23b50eb 100755 --- a/third_party/protobuf/python/google/protobuf/internal/text_format_test.py +++ b/third_party/protobuf/python/google/protobuf/internal/text_format_test.py @@ -94,6 +94,28 @@ class TextFormatTest(unittest.TestCase): ' }\n' '}\n') + def testPrintBadEnumValue(self): + message = unittest_pb2.TestAllTypes() + message.optional_nested_enum = 100 + message.optional_foreign_enum = 101 + message.optional_import_enum = 102 + self.CompareToGoldenText( + text_format.MessageToString(message), + 'optional_nested_enum: 100\n' + 'optional_foreign_enum: 101\n' + 'optional_import_enum: 102\n') + + def testPrintBadEnumValueExtensions(self): + message = unittest_pb2.TestAllExtensions() + message.Extensions[unittest_pb2.optional_nested_enum_extension] = 100 + message.Extensions[unittest_pb2.optional_foreign_enum_extension] = 101 + message.Extensions[unittest_pb2.optional_import_enum_extension] = 102 + self.CompareToGoldenText( + text_format.MessageToString(message), + '[protobuf_unittest.optional_nested_enum_extension]: 100\n' + '[protobuf_unittest.optional_foreign_enum_extension]: 101\n' + '[protobuf_unittest.optional_import_enum_extension]: 102\n') + def testPrintExotic(self): message = unittest_pb2.TestAllTypes() message.repeated_int64.append(-9223372036854775808) @@ -399,6 +421,14 @@ class TextFormatTest(unittest.TestCase): 'has no value with number 100.'), text_format.Merge, text, message) + def testMergeBadIntValue(self): + message = unittest_pb2.TestAllTypes() + text = 'optional_int32: bork' + self.assertRaisesWithMessage( + text_format.ParseError, + ('1:17 : Couldn\'t parse integer: bork'), + text_format.Merge, text, message) + def assertRaisesWithMessage(self, e_class, e, func, *args, **kwargs): """Same as assertRaises, but also compares the exception message.""" if hasattr(e_class, '__name__'): @@ -408,7 +438,7 @@ class TextFormatTest(unittest.TestCase): try: func(*args, **kwargs) - except e_class, expr: + except e_class as expr: if str(expr) != e: msg = '%s raised, but with wrong message: "%s" instead of "%s"' raise self.failureException(msg % (exc_name, @@ -427,7 +457,7 @@ class TokenizerTest(unittest.TestCase): 'identifiER_4 : 1.1e+2 ID5:-0.23 ID6:\'aaaa\\\'bbbb\'\n' 'ID7 : "aa\\"bb"\n\n\n\n ID8: {A:inf B:-inf C:true D:false}\n' 'ID9: 22 ID10: -111111111111111111 ID11: -22\n' - 'ID12: 2222222222222222222 ' + 'ID12: 2222222222222222222 ID13: 1.23456f ID14: 1.2e+2f ' 'false_bool: 0 true_BOOL:t \n true_bool1: 1 false_BOOL1:f ' ) tokenizer = text_format._Tokenizer(text) methods = [(tokenizer.ConsumeIdentifier, 'identifier1'), @@ -456,10 +486,10 @@ class TokenizerTest(unittest.TestCase): '{', (tokenizer.ConsumeIdentifier, 'A'), ':', - (tokenizer.ConsumeFloat, text_format._INFINITY), + (tokenizer.ConsumeFloat, float('inf')), (tokenizer.ConsumeIdentifier, 'B'), ':', - (tokenizer.ConsumeFloat, -text_format._INFINITY), + (tokenizer.ConsumeFloat, -float('inf')), (tokenizer.ConsumeIdentifier, 'C'), ':', (tokenizer.ConsumeBool, True), @@ -479,6 +509,12 @@ class TokenizerTest(unittest.TestCase): (tokenizer.ConsumeIdentifier, 'ID12'), ':', (tokenizer.ConsumeUint64, 2222222222222222222), + (tokenizer.ConsumeIdentifier, 'ID13'), + ':', + (tokenizer.ConsumeFloat, 1.23456), + (tokenizer.ConsumeIdentifier, 'ID14'), + ':', + (tokenizer.ConsumeFloat, 1.2e+2), (tokenizer.ConsumeIdentifier, 'false_bool'), ':', (tokenizer.ConsumeBool, False), @@ -556,16 +592,6 @@ class TokenizerTest(unittest.TestCase): tokenizer = text_format._Tokenizer(text) self.assertRaises(text_format.ParseError, tokenizer.ConsumeBool) - def testInfNan(self): - # Make sure our infinity and NaN definitions are sound. - self.assertEquals(float, type(text_format._INFINITY)) - self.assertEquals(float, type(text_format._NAN)) - self.assertTrue(text_format._NAN != text_format._NAN) - - inf_times_zero = text_format._INFINITY * 0 - self.assertTrue(inf_times_zero != inf_times_zero) - self.assertTrue(text_format._INFINITY > 0) - if __name__ == '__main__': unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/internal/unknown_fields_test.py b/third_party/protobuf/python/google/protobuf/internal/unknown_fields_test.py new file mode 100644 index 0000000..84984b4 --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/internal/unknown_fields_test.py @@ -0,0 +1,170 @@ +#! /usr/bin/python +# -*- coding: utf-8 -*- +# +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Test for preservation of unknown fields in the pure Python implementation.""" + +__author__ = 'bohdank@google.com (Bohdan Koval)' + +import unittest +from google.protobuf import unittest_mset_pb2 +from google.protobuf import unittest_pb2 +from google.protobuf.internal import encoder +from google.protobuf.internal import test_util +from google.protobuf.internal import type_checkers + + +class UnknownFieldsTest(unittest.TestCase): + + def setUp(self): + self.descriptor = unittest_pb2.TestAllTypes.DESCRIPTOR + self.all_fields = unittest_pb2.TestAllTypes() + test_util.SetAllFields(self.all_fields) + self.all_fields_data = self.all_fields.SerializeToString() + self.empty_message = unittest_pb2.TestEmptyMessage() + self.empty_message.ParseFromString(self.all_fields_data) + self.unknown_fields = self.empty_message._unknown_fields + + def GetField(self, name): + field_descriptor = self.descriptor.fields_by_name[name] + wire_type = type_checkers.FIELD_TYPE_TO_WIRE_TYPE[field_descriptor.type] + field_tag = encoder.TagBytes(field_descriptor.number, wire_type) + for tag_bytes, value in self.unknown_fields: + if tag_bytes == field_tag: + decoder = unittest_pb2.TestAllTypes._decoders_by_tag[tag_bytes] + result_dict = {} + decoder(value, 0, len(value), self.all_fields, result_dict) + return result_dict[field_descriptor] + + def testVarint(self): + value = self.GetField('optional_int32') + self.assertEqual(self.all_fields.optional_int32, value) + + def testFixed32(self): + value = self.GetField('optional_fixed32') + self.assertEqual(self.all_fields.optional_fixed32, value) + + def testFixed64(self): + value = self.GetField('optional_fixed64') + self.assertEqual(self.all_fields.optional_fixed64, value) + + def testLengthDelimited(self): + value = self.GetField('optional_string') + self.assertEqual(self.all_fields.optional_string, value) + + def testGroup(self): + value = self.GetField('optionalgroup') + self.assertEqual(self.all_fields.optionalgroup, value) + + def testSerialize(self): + data = self.empty_message.SerializeToString() + + # Don't use assertEqual because we don't want to dump raw binary data to + # stdout. + self.assertTrue(data == self.all_fields_data) + + def testCopyFrom(self): + message = unittest_pb2.TestEmptyMessage() + message.CopyFrom(self.empty_message) + self.assertEqual(self.unknown_fields, message._unknown_fields) + + def testMergeFrom(self): + message = unittest_pb2.TestAllTypes() + message.optional_int32 = 1 + message.optional_uint32 = 2 + source = unittest_pb2.TestEmptyMessage() + source.ParseFromString(message.SerializeToString()) + + message.ClearField('optional_int32') + message.optional_int64 = 3 + message.optional_uint32 = 4 + destination = unittest_pb2.TestEmptyMessage() + destination.ParseFromString(message.SerializeToString()) + unknown_fields = destination._unknown_fields[:] + + destination.MergeFrom(source) + self.assertEqual(unknown_fields + source._unknown_fields, + destination._unknown_fields) + + def testClear(self): + self.empty_message.Clear() + self.assertEqual(0, len(self.empty_message._unknown_fields)) + + def testByteSize(self): + self.assertEqual(self.all_fields.ByteSize(), self.empty_message.ByteSize()) + + def testUnknownExtensions(self): + message = unittest_pb2.TestEmptyMessageWithExtensions() + message.ParseFromString(self.all_fields_data) + self.assertEqual(self.empty_message._unknown_fields, + message._unknown_fields) + + def testListFields(self): + # Make sure ListFields doesn't return unknown fields. + self.assertEqual(0, len(self.empty_message.ListFields())) + + def testSerializeMessageSetWireFormatUnknownExtension(self): + # Create a message using the message set wire format with an unknown + # message. + raw = unittest_mset_pb2.RawMessageSet() + + # Add an unknown extension. + item = raw.item.add() + item.type_id = 1545009 + message1 = unittest_mset_pb2.TestMessageSetExtension1() + message1.i = 12345 + item.message = message1.SerializeToString() + + serialized = raw.SerializeToString() + + # Parse message using the message set wire format. + proto = unittest_mset_pb2.TestMessageSet() + proto.MergeFromString(serialized) + + # Verify that the unknown extension is serialized unchanged + reserialized = proto.SerializeToString() + new_raw = unittest_mset_pb2.RawMessageSet() + new_raw.MergeFromString(reserialized) + self.assertEqual(raw, new_raw) + + def testEquals(self): + message = unittest_pb2.TestEmptyMessage() + message.ParseFromString(self.all_fields_data) + self.assertEqual(self.empty_message, message) + + self.all_fields.ClearField('optional_string') + message.ParseFromString(self.all_fields.SerializeToString()) + self.assertNotEqual(self.empty_message, message) + + +if __name__ == '__main__': + unittest.main() diff --git a/third_party/protobuf/python/google/protobuf/message.py b/third_party/protobuf/python/google/protobuf/message.py index 6f19f85..6ec2f8b 100755 --- a/third_party/protobuf/python/google/protobuf/message.py +++ b/third_party/protobuf/python/google/protobuf/message.py @@ -73,6 +73,7 @@ class Message(object): return clone def __eq__(self, other_msg): + """Recursively compares two messages by value and structure.""" raise NotImplementedError def __ne__(self, other_msg): @@ -83,9 +84,11 @@ class Message(object): raise TypeError('unhashable object') def __str__(self): + """Outputs a human-readable representation of the message.""" raise NotImplementedError def __unicode__(self): + """Outputs a human-readable representation of the message.""" raise NotImplementedError def MergeFrom(self, other_msg): @@ -266,3 +269,12 @@ class Message(object): via a previous _SetListener() call. """ raise NotImplementedError + + def __getstate__(self): + """Support the pickle protocol.""" + return dict(serialized=self.SerializePartialToString()) + + def __setstate__(self, state): + """Support the pickle protocol.""" + self.__init__() + self.ParseFromString(state['serialized']) diff --git a/third_party/protobuf/python/google/protobuf/message_factory.py b/third_party/protobuf/python/google/protobuf/message_factory.py new file mode 100644 index 0000000..36e2fef --- /dev/null +++ b/third_party/protobuf/python/google/protobuf/message_factory.py @@ -0,0 +1,113 @@ +# Protocol Buffers - Google's data interchange format +# Copyright 2008 Google Inc. All rights reserved. +# http://code.google.com/p/protobuf/ +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +"""Provides a factory class for generating dynamic messages.""" + +__author__ = 'matthewtoia@google.com (Matt Toia)' + +from google.protobuf import descriptor_database +from google.protobuf import descriptor_pool +from google.protobuf import message +from google.protobuf import reflection + + +class MessageFactory(object): + """Factory for creating Proto2 messages from descriptors in a pool.""" + + def __init__(self): + """Initializes a new factory.""" + self._classes = {} + + def GetPrototype(self, descriptor): + """Builds a proto2 message class based on the passed in descriptor. + + Passing a descriptor with a fully qualified name matching a previous + invocation will cause the same class to be returned. + + Args: + descriptor: The descriptor to build from. + + Returns: + A class describing the passed in descriptor. + """ + + if descriptor.full_name not in self._classes: + result_class = reflection.GeneratedProtocolMessageType( + descriptor.name.encode('ascii', 'ignore'), + (message.Message,), + {'DESCRIPTOR': descriptor}) + self._classes[descriptor.full_name] = result_class + for field in descriptor.fields: + if field.message_type: + self.GetPrototype(field.message_type) + return self._classes[descriptor.full_name] + + +_DB = descriptor_database.DescriptorDatabase() +_POOL = descriptor_pool.DescriptorPool(_DB) +_FACTORY = MessageFactory() + + +def GetMessages(file_protos): + """Builds a dictionary of all the messages available in a set of files. + + Args: + file_protos: A sequence of file protos to build messages out of. + + Returns: + A dictionary containing all the message types in the files mapping the + fully qualified name to a Message subclass for the descriptor. + """ + + result = {} + for file_proto in file_protos: + _DB.Add(file_proto) + for file_proto in file_protos: + for desc in _GetAllDescriptors(file_proto.message_type, file_proto.package): + result[desc.full_name] = _FACTORY.GetPrototype(desc) + return result + + +def _GetAllDescriptors(desc_protos, package): + """Gets all levels of nested message types as a flattened list of descriptors. + + Args: + desc_protos: The descriptor protos to process. + package: The package where the protos are defined. + + Yields: + Each message descriptor for each nested type. + """ + + for desc_proto in desc_protos: + name = '.'.join((package, desc_proto.name)) + yield _POOL.FindMessageTypeByName(name) + for nested_desc in _GetAllDescriptors(desc_proto.nested_type, name): + yield nested_desc diff --git a/third_party/protobuf/python/google/protobuf/pyext/python-proto2.cc b/third_party/protobuf/python/google/protobuf/pyext/python-proto2.cc index 181d7e9..6275341 100644 --- a/third_party/protobuf/python/google/protobuf/pyext/python-proto2.cc +++ b/third_party/protobuf/python/google/protobuf/pyext/python-proto2.cc @@ -207,9 +207,9 @@ static PyMethodDef CMessageMethods[] = { "Clears and sets the values of a repeated scalar field."), CMETHOD(ByteSize, METH_NOARGS, "Returns the size of the message in bytes."), - CMETHOD(Clear, METH_NOARGS, + CMETHOD(Clear, METH_O, "Clears a protocol message."), - CMETHOD(ClearField, METH_O, + CMETHOD(ClearField, METH_VARARGS, "Clears a protocol message field by name."), CMETHOD(ClearFieldByDescriptor, METH_O, "Clears a protocol message field by descriptor."), @@ -274,7 +274,7 @@ static PyMemberDef CMessageMembers[] = { PyTypeObject CMessage_Type = { PyObject_HEAD_INIT(&PyType_Type) 0, - C("google3.net.google.protobuf.python.internal." + C("google.protobuf.internal." "_net_proto2___python." "CMessage"), // tp_name sizeof(CMessage), // tp_basicsize @@ -319,14 +319,12 @@ PyTypeObject CMessage_Type = { // ------ Helper Functions: static void FormatTypeError(PyObject* arg, char* expected_types) { - PyObject* s = PyObject_Str(PyObject_Type(arg)); - PyObject* repr = PyObject_Repr(PyObject_Type(arg)); + PyObject* repr = PyObject_Repr(arg); PyErr_Format(PyExc_TypeError, "%.100s has type %.100s, but expected one of: %s", PyString_AS_STRING(repr), - PyString_AS_STRING(s), + arg->ob_type->tp_name, expected_types); - Py_DECREF(s); Py_DECREF(repr); } @@ -398,6 +396,28 @@ static const google::protobuf::Message* CreateMessage(const char* message_type) return global_message_factory->GetPrototype(descriptor); } +static void ReleaseSubMessage(google::protobuf::Message* message, + const google::protobuf::FieldDescriptor* field_descriptor, + CMessage* child_cmessage) { + Message* released_message = message->GetReflection()->ReleaseMessage( + message, field_descriptor, global_message_factory); + GOOGLE_DCHECK(child_cmessage->message != NULL); + // ReleaseMessage will return NULL which differs from + // child_cmessage->message, if the field does not exist. In this case, + // the latter points to the default instance via a const_cast<>, so we + // have to reset it to a new mutable object since we are taking ownership. + if (released_message == NULL) { + const Message* prototype = global_message_factory->GetPrototype( + child_cmessage->message->GetDescriptor()); + GOOGLE_DCHECK(prototype != NULL); + child_cmessage->message = prototype->New(); + } + child_cmessage->parent = NULL; + child_cmessage->parent_field = NULL; + child_cmessage->free_message = true; + child_cmessage->read_only = false; +} + static bool CheckAndSetString( PyObject* arg, google::protobuf::Message* message, const google::protobuf::FieldDescriptor* descriptor, @@ -407,6 +427,9 @@ static bool CheckAndSetString( GOOGLE_DCHECK(descriptor->type() == google::protobuf::FieldDescriptor::TYPE_STRING || descriptor->type() == google::protobuf::FieldDescriptor::TYPE_BYTES); if (descriptor->type() == google::protobuf::FieldDescriptor::TYPE_STRING) { +#else + if (descriptor->file()->options().cc_api_version() == 2 && + descriptor->type() == google::protobuf::FieldDescriptor::TYPE_STRING) { if (!PyString_Check(arg) && !PyUnicode_Check(arg)) { FormatTypeError(arg, "str, unicode"); return false; @@ -434,6 +457,9 @@ static bool CheckAndSetString( PyObject* encoded_string = NULL; if (descriptor->type() == google::protobuf::FieldDescriptor::TYPE_STRING) { +#else + if (descriptor->file()->options().cc_api_version() == 2 && + descriptor->type() == google::protobuf::FieldDescriptor::TYPE_STRING) { if (PyString_Check(arg)) { encoded_string = PyString_AsEncodedObject(arg, "utf-8", NULL); } else { @@ -504,8 +530,6 @@ static void AssureWritable(CMessage* self) { self->message = reflection->MutableMessage( message, self->parent_field->descriptor, global_message_factory); self->read_only = false; - self->parent = NULL; - self->parent_field = NULL; } static PyObject* InternalGetScalar( @@ -955,9 +979,41 @@ static void CMessageDealloc(CMessage* self) { // ------ Methods: -static PyObject* CMessage_Clear(CMessage* self, PyObject* args) { +static PyObject* CMessage_Clear(CMessage* self, PyObject* arg) { AssureWritable(self); - self->message->Clear(); + google::protobuf::Message* message = self->message; + + // This block of code is equivalent to the following: + // for cfield_descriptor, child_cmessage in arg: + // ReleaseSubMessage(cfield_descriptor, child_cmessage) + if (!PyList_Check(arg)) { + PyErr_SetString(PyExc_TypeError, "Must be a list"); + return NULL; + } + PyObject* messages_to_clear = arg; + Py_ssize_t num_messages_to_clear = PyList_GET_SIZE(messages_to_clear); + for(int i = 0; i < num_messages_to_clear; ++i) { + PyObject* message_tuple = PyList_GET_ITEM(messages_to_clear, i); + if (!PyTuple_Check(message_tuple) || PyTuple_GET_SIZE(message_tuple) != 2) { + PyErr_SetString(PyExc_TypeError, "Must be a tuple of size 2"); + return NULL; + } + + PyObject* py_cfield_descriptor = PyTuple_GET_ITEM(message_tuple, 0); + PyObject* py_child_cmessage = PyTuple_GET_ITEM(message_tuple, 1); + if (!PyObject_TypeCheck(py_cfield_descriptor, &CFieldDescriptor_Type) || + !PyObject_TypeCheck(py_child_cmessage, &CMessage_Type)) { + PyErr_SetString(PyExc_ValueError, "Invalid Tuple"); + return NULL; + } + + CFieldDescriptor* cfield_descriptor = reinterpret_cast<CFieldDescriptor *>( + py_cfield_descriptor); + CMessage* child_cmessage = reinterpret_cast<CMessage *>(py_child_cmessage); + ReleaseSubMessage(message, cfield_descriptor->descriptor, child_cmessage); + } + + message->Clear(); Py_RETURN_NONE; } @@ -1039,9 +1095,11 @@ static PyObject* CMessage_ClearFieldByDescriptor( Py_RETURN_NONE; } -static PyObject* CMessage_ClearField(CMessage* self, PyObject* arg) { +static PyObject* CMessage_ClearField(CMessage* self, PyObject* args) { char* field_name; - if (PyString_AsStringAndSize(arg, &field_name, NULL) < 0) { + CMessage* child_cmessage = NULL; + if (!PyArg_ParseTuple(args, C("s|O!:ClearField"), &field_name, + &CMessage_Type, &child_cmessage)) { return NULL; } @@ -1054,7 +1112,11 @@ static PyObject* CMessage_ClearField(CMessage* self, PyObject* arg) { return NULL; } - message->GetReflection()->ClearField(message, field_descriptor); + if (child_cmessage != NULL && !FIELD_IS_REPEATED(field_descriptor)) { + ReleaseSubMessage(message, field_descriptor, child_cmessage); + } else { + message->GetReflection()->ClearField(message, field_descriptor); + } Py_RETURN_NONE; } @@ -1313,6 +1375,7 @@ static PyObject* CMessage_MergeFromString(CMessage* self, PyObject* arg) { AssureWritable(self); google::protobuf::io::CodedInputStream input( reinterpret_cast<const uint8*>(data), data_length); + input.SetExtensionRegistry(GetDescriptorPool(), global_message_factory); bool success = self->message->MergePartialFromCodedStream(&input); if (success) { return PyInt_FromLong(self->message->ByteSize()); diff --git a/third_party/protobuf/python/google/protobuf/pyext/python_descriptor.cc b/third_party/protobuf/python/google/protobuf/pyext/python_descriptor.cc index fb87bad..5e3e9ea 100644 --- a/third_party/protobuf/python/google/protobuf/pyext/python_descriptor.cc +++ b/third_party/protobuf/python/google/protobuf/pyext/python_descriptor.cc @@ -31,6 +31,7 @@ // Author: petar@google.com (Petar Petrov) #include <Python.h> +#include <string> #include <google/protobuf/pyext/python_descriptor.h> #include <google/protobuf/descriptor.pb.h> @@ -41,6 +42,7 @@ namespace google { namespace protobuf { namespace python { + static void CFieldDescriptorDealloc(CFieldDescriptor* self); static google::protobuf::DescriptorPool* g_descriptor_pool = NULL; @@ -93,7 +95,7 @@ static PyGetSetDef CFieldDescriptorGetters[] = { PyTypeObject CFieldDescriptor_Type = { PyObject_HEAD_INIT(&PyType_Type) 0, - C("google3.net.google.protobuf.python.internal." + C("google.protobuf.internal." "_net_proto2___python." "CFieldDescriptor"), // tp_name sizeof(CFieldDescriptor), // tp_basicsize @@ -181,6 +183,8 @@ static PyObject* CDescriptorPool_FindFieldByName( const google::protobuf::FieldDescriptor* field_descriptor = NULL; field_descriptor = self->pool->FindFieldByName(full_field_name); + + if (field_descriptor == NULL) { PyErr_Format(PyExc_TypeError, "Couldn't find field %.200s", full_field_name); @@ -223,7 +227,7 @@ static PyMethodDef CDescriptorPoolMethods[] = { PyTypeObject CDescriptorPool_Type = { PyObject_HEAD_INIT(&PyType_Type) 0, - C("google3.net.google.protobuf.python.internal." + C("google.protobuf.internal." "_net_proto2___python." "CFieldDescriptor"), // tp_name sizeof(CDescriptorPool), // tp_basicsize @@ -301,7 +305,6 @@ PyObject* Python_BuildFile(PyObject* ignored, PyObject* arg) { return NULL; } - // If this file is already in the generated pool, don't add it again. if (google::protobuf::DescriptorPool::generated_pool()->FindFileByName( file_proto.name()) != NULL) { Py_RETURN_NONE; diff --git a/third_party/protobuf/python/google/protobuf/reflection.py b/third_party/protobuf/python/google/protobuf/reflection.py index 1373c88..9570fd5 100755 --- a/third_party/protobuf/python/google/protobuf/reflection.py +++ b/third_party/protobuf/python/google/protobuf/reflection.py @@ -50,13 +50,20 @@ __author__ = 'robinson@google.com (Will Robinson)' from google.protobuf.internal import api_implementation from google.protobuf import descriptor as descriptor_mod +from google.protobuf import message + _FieldDescriptor = descriptor_mod.FieldDescriptor if api_implementation.Type() == 'cpp': - from google.protobuf.internal import cpp_message - _NewMessage = cpp_message.NewMessage - _InitMessage = cpp_message.InitMessage + if api_implementation.Version() == 2: + from google.protobuf.internal.cpp import cpp_message + _NewMessage = cpp_message.NewMessage + _InitMessage = cpp_message.InitMessage + else: + from google.protobuf.internal import cpp_message + _NewMessage = cpp_message.NewMessage + _InitMessage = cpp_message.InitMessage else: from google.protobuf.internal import python_message _NewMessage = python_message.NewMessage @@ -112,7 +119,7 @@ class GeneratedProtocolMessageType(type): Newly-allocated class. """ descriptor = dictionary[GeneratedProtocolMessageType._DESCRIPTOR_KEY] - _NewMessage(descriptor, dictionary) + bases = _NewMessage(bases, descriptor, dictionary) superclass = super(GeneratedProtocolMessageType, cls) new_class = superclass.__new__(cls, name, bases, dictionary) @@ -140,3 +147,23 @@ class GeneratedProtocolMessageType(type): _InitMessage(descriptor, cls) superclass = super(GeneratedProtocolMessageType, cls) superclass.__init__(name, bases, dictionary) + + +def ParseMessage(descriptor, byte_str): + """Generate a new Message instance from this Descriptor and a byte string. + + Args: + descriptor: Protobuf Descriptor object + byte_str: Serialized protocol buffer byte string + + Returns: + Newly created protobuf Message object. + """ + + class _ResultClass(message.Message): + __metaclass__ = GeneratedProtocolMessageType + DESCRIPTOR = descriptor + + new_msg = _ResultClass() + new_msg.ParseFromString(byte_str) + return new_msg diff --git a/third_party/protobuf/python/google/protobuf/text_format.py b/third_party/protobuf/python/google/protobuf/text_format.py index c3a1cf6..0714c39 100755 --- a/third_party/protobuf/python/google/protobuf/text_format.py +++ b/third_party/protobuf/python/google/protobuf/text_format.py @@ -43,10 +43,12 @@ __all__ = [ 'MessageToString', 'PrintMessage', 'PrintField', 'PrintFieldValue', 'Merge' ] -# Infinity and NaN are not explicitly supported by Python pre-2.6, and -# float('inf') does not work on Windows (pre-2.6). -_INFINITY = 1e10000 # overflows, thus will actually be infinity. -_NAN = _INFINITY * 0 +_INTEGER_CHECKERS = (type_checkers.Uint32ValueChecker(), + type_checkers.Int32ValueChecker(), + type_checkers.Uint64ValueChecker(), + type_checkers.Int64ValueChecker()) +_FLOAT_INFINITY = re.compile('-?inf(?:inity)?f?', re.IGNORECASE) +_FLOAT_NAN = re.compile('nanf?', re.IGNORECASE) class ParseError(Exception): @@ -120,7 +122,11 @@ def PrintFieldValue(field, value, out, indent=0, PrintMessage(value, out, indent + 2, as_utf8, as_one_line) out.write(' ' * indent + '}') elif field.cpp_type == descriptor.FieldDescriptor.CPPTYPE_ENUM: - out.write(field.enum_type.values_by_number[value].name) + enum_value = field.enum_type.values_by_number.get(value, None) + if enum_value is not None: + out.write(enum_value.name) + else: + out.write(str(value)) elif field.cpp_type == descriptor.FieldDescriptor.CPPTYPE_STRING: out.write('\"') if type(value) is unicode: @@ -271,24 +277,7 @@ def _MergeScalarField(tokenizer, message, field): elif field.type == descriptor.FieldDescriptor.TYPE_BYTES: value = tokenizer.ConsumeByteString() elif field.type == descriptor.FieldDescriptor.TYPE_ENUM: - # Enum can be specified by a number (the enum value), or by - # a string literal (the enum name). - enum_descriptor = field.enum_type - if tokenizer.LookingAtInteger(): - number = tokenizer.ConsumeInt32() - enum_value = enum_descriptor.values_by_number.get(number, None) - if enum_value is None: - raise tokenizer.ParseErrorPreviousToken( - 'Enum type "%s" has no value with number %d.' % ( - enum_descriptor.full_name, number)) - else: - identifier = tokenizer.ConsumeIdentifier() - enum_value = enum_descriptor.values_by_name.get(identifier, None) - if enum_value is None: - raise tokenizer.ParseErrorPreviousToken( - 'Enum type "%s" has no value named %s.' % ( - enum_descriptor.full_name, identifier)) - value = enum_value.number + value = tokenizer.ConsumeEnum(field) else: raise RuntimeError('Unknown field type %d' % field.type) @@ -320,12 +309,6 @@ class _Tokenizer(object): '\"([^\"\n\\\\]|\\\\.)*(\"|\\\\?$)|' # a double-quoted string '\'([^\'\n\\\\]|\\\\.)*(\'|\\\\?$)') # a single-quoted string _IDENTIFIER = re.compile('\w+') - _INTEGER_CHECKERS = [type_checkers.Uint32ValueChecker(), - type_checkers.Int32ValueChecker(), - type_checkers.Uint64ValueChecker(), - type_checkers.Int64ValueChecker()] - _FLOAT_INFINITY = re.compile('-?inf(inity)?f?', re.IGNORECASE) - _FLOAT_NAN = re.compile("nanf?", re.IGNORECASE) def __init__(self, text_message): self._text_message = text_message @@ -394,17 +377,6 @@ class _Tokenizer(object): if not self.TryConsume(token): raise self._ParseError('Expected "%s".' % token) - def LookingAtInteger(self): - """Checks if the current token is an integer. - - Returns: - True iff the current token is an integer. - """ - if not self.token: - return False - c = self.token[0] - return (c >= '0' and c <= '9') or c == '-' or c == '+' - def ConsumeIdentifier(self): """Consumes protocol message field identifier. @@ -430,9 +402,9 @@ class _Tokenizer(object): ParseError: If a signed 32bit integer couldn't be consumed. """ try: - result = self._ParseInteger(self.token, is_signed=True, is_long=False) + result = ParseInteger(self.token, is_signed=True, is_long=False) except ValueError, e: - raise self._IntegerParseError(e) + raise self._ParseError(str(e)) self.NextToken() return result @@ -446,9 +418,9 @@ class _Tokenizer(object): ParseError: If an unsigned 32bit integer couldn't be consumed. """ try: - result = self._ParseInteger(self.token, is_signed=False, is_long=False) + result = ParseInteger(self.token, is_signed=False, is_long=False) except ValueError, e: - raise self._IntegerParseError(e) + raise self._ParseError(str(e)) self.NextToken() return result @@ -462,9 +434,9 @@ class _Tokenizer(object): ParseError: If a signed 64bit integer couldn't be consumed. """ try: - result = self._ParseInteger(self.token, is_signed=True, is_long=True) + result = ParseInteger(self.token, is_signed=True, is_long=True) except ValueError, e: - raise self._IntegerParseError(e) + raise self._ParseError(str(e)) self.NextToken() return result @@ -478,9 +450,9 @@ class _Tokenizer(object): ParseError: If an unsigned 64bit integer couldn't be consumed. """ try: - result = self._ParseInteger(self.token, is_signed=False, is_long=True) + result = ParseInteger(self.token, is_signed=False, is_long=True) except ValueError, e: - raise self._IntegerParseError(e) + raise self._ParseError(str(e)) self.NextToken() return result @@ -493,21 +465,10 @@ class _Tokenizer(object): Raises: ParseError: If a floating point number couldn't be consumed. """ - text = self.token - if self._FLOAT_INFINITY.match(text): - self.NextToken() - if text.startswith('-'): - return -_INFINITY - return _INFINITY - - if self._FLOAT_NAN.match(text): - self.NextToken() - return _NAN - try: - result = float(text) + result = ParseFloat(self.token) except ValueError, e: - raise self._FloatParseError(e) + raise self._ParseError(str(e)) self.NextToken() return result @@ -520,14 +481,12 @@ class _Tokenizer(object): Raises: ParseError: If a boolean value couldn't be consumed. """ - if self.token in ('true', 't', '1'): - self.NextToken() - return True - elif self.token in ('false', 'f', '0'): - self.NextToken() - return False - else: - raise self._ParseError('Expected "true" or "false".') + try: + result = ParseBool(self.token) + except ValueError, e: + raise self._ParseError(str(e)) + self.NextToken() + return result def ConsumeString(self): """Consumes a string value. @@ -567,7 +526,7 @@ class _Tokenizer(object): """ text = self.token if len(text) < 1 or text[0] not in ('\'', '"'): - raise self._ParseError('Exptected string.') + raise self._ParseError('Expected string.') if len(text) < 2 or text[-1] != text[0]: raise self._ParseError('String missing ending quote.') @@ -579,36 +538,12 @@ class _Tokenizer(object): self.NextToken() return result - def _ParseInteger(self, text, is_signed=False, is_long=False): - """Parses an integer. - - Args: - text: The text to parse. - is_signed: True if a signed integer must be parsed. - is_long: True if a long integer must be parsed. - - Returns: - The integer value. - - Raises: - ValueError: Thrown Iff the text is not a valid integer. - """ - pos = 0 - if text.startswith('-'): - pos += 1 - - base = 10 - if text.startswith('0x', pos) or text.startswith('0X', pos): - base = 16 - elif text.startswith('0', pos): - base = 8 - - # Do the actual parsing. Exception handling is propagated to caller. - result = int(text, base) - - # Check if the integer is sane. Exceptions handled by callers. - checker = self._INTEGER_CHECKERS[2 * int(is_long) + int(is_signed)] - checker.CheckValue(result) + def ConsumeEnum(self, field): + try: + result = ParseEnum(field, self.token) + except ValueError, e: + raise self._ParseError(str(e)) + self.NextToken() return result def ParseErrorPreviousToken(self, message): @@ -626,13 +561,7 @@ class _Tokenizer(object): def _ParseError(self, message): """Creates and *returns* a ParseError for the current token.""" return ParseError('%d:%d : %s' % ( - self._line + 1, self._column - len(self.token) + 1, message)) - - def _IntegerParseError(self, e): - return self._ParseError('Couldn\'t parse integer: ' + str(e)) - - def _FloatParseError(self, e): - return self._ParseError('Couldn\'t parse number: ' + str(e)) + self._line + 1, self._column + 1, message)) def _StringParseError(self, e): return self._ParseError('Couldn\'t parse string: ' + str(e)) @@ -689,3 +618,117 @@ def _CUnescape(text): # allow single-digit hex escapes (like '\xf'). result = _CUNESCAPE_HEX.sub(ReplaceHex, text) return result.decode('string_escape') + + +def ParseInteger(text, is_signed=False, is_long=False): + """Parses an integer. + + Args: + text: The text to parse. + is_signed: True if a signed integer must be parsed. + is_long: True if a long integer must be parsed. + + Returns: + The integer value. + + Raises: + ValueError: Thrown Iff the text is not a valid integer. + """ + # Do the actual parsing. Exception handling is propagated to caller. + try: + result = int(text, 0) + except ValueError: + raise ValueError('Couldn\'t parse integer: %s' % text) + + # Check if the integer is sane. Exceptions handled by callers. + checker = _INTEGER_CHECKERS[2 * int(is_long) + int(is_signed)] + checker.CheckValue(result) + return result + + +def ParseFloat(text): + """Parse a floating point number. + + Args: + text: Text to parse. + + Returns: + The number parsed. + + Raises: + ValueError: If a floating point number couldn't be parsed. + """ + try: + # Assume Python compatible syntax. + return float(text) + except ValueError: + # Check alternative spellings. + if _FLOAT_INFINITY.match(text): + if text[0] == '-': + return float('-inf') + else: + return float('inf') + elif _FLOAT_NAN.match(text): + return float('nan') + else: + # assume '1.0f' format + try: + return float(text.rstrip('f')) + except ValueError: + raise ValueError('Couldn\'t parse float: %s' % text) + + +def ParseBool(text): + """Parse a boolean value. + + Args: + text: Text to parse. + + Returns: + Boolean values parsed + + Raises: + ValueError: If text is not a valid boolean. + """ + if text in ('true', 't', '1'): + return True + elif text in ('false', 'f', '0'): + return False + else: + raise ValueError('Expected "true" or "false".') + + +def ParseEnum(field, value): + """Parse an enum value. + + The value can be specified by a number (the enum value), or by + a string literal (the enum name). + + Args: + field: Enum field descriptor. + value: String value. + + Returns: + Enum value number. + + Raises: + ValueError: If the enum value could not be parsed. + """ + enum_descriptor = field.enum_type + try: + number = int(value, 0) + except ValueError: + # Identifier. + enum_value = enum_descriptor.values_by_name.get(value, None) + if enum_value is None: + raise ValueError( + 'Enum type "%s" has no value named %s.' % ( + enum_descriptor.full_name, value)) + else: + # Numeric value. + enum_value = enum_descriptor.values_by_number.get(number, None) + if enum_value is None: + raise ValueError( + 'Enum type "%s" has no value with number %d.' % ( + enum_descriptor.full_name, number)) + return enum_value.number diff --git a/third_party/protobuf/python/setup.py b/third_party/protobuf/python/setup.py index 7d9714e..fbe2766 100755 --- a/third_party/protobuf/python/setup.py +++ b/third_party/protobuf/python/setup.py @@ -1,17 +1,27 @@ #! /usr/bin/python # # See README for usage instructions. +import sys +import os +import subprocess # We must use setuptools, not distutils, because we need to use the # namespace_packages option for the "google" package. -from ez_setup import use_setuptools -use_setuptools() - -from setuptools import setup, Extension +try: + from setuptools import setup, Extension +except ImportError: + try: + from ez_setup import use_setuptools + use_setuptools() + from setuptools import setup, Extension + except ImportError: + sys.stderr.write( + "Could not import setuptools; make sure you have setuptools or " + "ez_setup installed.\n") + raise +from distutils.command.clean import clean as _clean +from distutils.command.build_py import build_py as _build_py from distutils.spawn import find_executable -import sys -import os -import subprocess maintainer_email = "protobuf@googlegroups.com" @@ -34,15 +44,15 @@ def generate_proto(source): output = source.replace(".proto", "_pb2.py").replace("../src/", "") - if not os.path.exists(source): - print "Can't find required file: " + source - sys.exit(-1) - if (not os.path.exists(output) or (os.path.exists(source) and os.path.getmtime(source) > os.path.getmtime(output))): print "Generating %s..." % output + if not os.path.exists(source): + sys.stderr.write("Can't find required file: %s\n" % source) + sys.exit(-1) + if protoc == None: sys.stderr.write( "protoc is not installed nor found in ../src. Please compile it " @@ -53,19 +63,26 @@ def generate_proto(source): if subprocess.call(protoc_command) != 0: sys.exit(-1) -def MakeTestSuite(): - # This is apparently needed on some systems to make sure that the tests - # work even if a previous version is already installed. - if 'google' in sys.modules: - del sys.modules['google'] - +def GenerateUnittestProtos(): generate_proto("../src/google/protobuf/unittest.proto") generate_proto("../src/google/protobuf/unittest_custom_options.proto") generate_proto("../src/google/protobuf/unittest_import.proto") + generate_proto("../src/google/protobuf/unittest_import_public.proto") generate_proto("../src/google/protobuf/unittest_mset.proto") generate_proto("../src/google/protobuf/unittest_no_generic_services.proto") + generate_proto("google/protobuf/internal/test_bad_identifiers.proto") generate_proto("google/protobuf/internal/more_extensions.proto") + generate_proto("google/protobuf/internal/more_extensions_dynamic.proto") generate_proto("google/protobuf/internal/more_messages.proto") + generate_proto("google/protobuf/internal/factory_test1.proto") + generate_proto("google/protobuf/internal/factory_test2.proto") + +def MakeTestSuite(): + # This is apparently needed on some systems to make sure that the tests + # work even if a previous version is already installed. + if 'google' in sys.modules: + del sys.modules['google'] + GenerateUnittestProtos() import unittest import google.protobuf.internal.generator_test as generator_test @@ -75,6 +92,14 @@ def MakeTestSuite(): as service_reflection_test import google.protobuf.internal.text_format_test as text_format_test import google.protobuf.internal.wire_format_test as wire_format_test + import google.protobuf.internal.unknown_fields_test as unknown_fields_test + import google.protobuf.internal.descriptor_database_test \ + as descriptor_database_test + import google.protobuf.internal.descriptor_pool_test as descriptor_pool_test + import google.protobuf.internal.message_factory_test as message_factory_test + import google.protobuf.internal.message_cpp_test as message_cpp_test + import google.protobuf.internal.reflection_cpp_generated_test \ + as reflection_cpp_generated_test loader = unittest.defaultTestLoader suite = unittest.TestSuite() @@ -88,22 +113,33 @@ def MakeTestSuite(): return suite -if __name__ == '__main__': - # TODO(kenton): Integrate this into setuptools somehow? - if len(sys.argv) >= 2 and sys.argv[1] == "clean": - # Delete generated _pb2.py files and .pyc files in the code tree. + +class clean(_clean): + def run(self): + # Delete generated files in the code tree. for (dirpath, dirnames, filenames) in os.walk("."): for filename in filenames: filepath = os.path.join(dirpath, filename) if filepath.endswith("_pb2.py") or filepath.endswith(".pyc") or \ - filepath.endswith(".so") or filepath.endswith(".o"): + filepath.endswith(".so") or filepath.endswith(".o") or \ + filepath.endswith('google/protobuf/compiler/__init__.py'): os.remove(filepath) - else: + # _clean is an old-style class, so super() doesn't work. + _clean.run(self) + +class build_py(_build_py): + def run(self): # Generate necessary .proto file if it doesn't exist. - # TODO(kenton): Maybe we should hook this into a distutils command? generate_proto("../src/google/protobuf/descriptor.proto") generate_proto("../src/google/protobuf/compiler/plugin.proto") + GenerateUnittestProtos() + # Make sure google.protobuf.compiler is a valid package. + open('google/protobuf/compiler/__init__.py', 'a').close() + # _build_py is an old-style class, so super() doesn't work. + _build_py.run(self) + +if __name__ == '__main__': ext_module_list = [] # C++ implementation extension @@ -137,10 +173,15 @@ if __name__ == '__main__': 'google.protobuf.descriptor_pb2', 'google.protobuf.compiler.plugin_pb2', 'google.protobuf.message', + 'google.protobuf.descriptor_database', + 'google.protobuf.descriptor_pool', + 'google.protobuf.message_factory', 'google.protobuf.reflection', 'google.protobuf.service', 'google.protobuf.service_reflection', 'google.protobuf.text_format' ], + cmdclass = { 'clean': clean, 'build_py': build_py }, + install_requires = ['setuptools'], ext_modules = ext_module_list, url = 'http://code.google.com/p/protobuf/', maintainer = maintainer_email, diff --git a/third_party/protobuf/src/Makefile.am b/third_party/protobuf/src/Makefile.am index db9e7af..172d0cd 100644 --- a/third_party/protobuf/src/Makefile.am +++ b/third_party/protobuf/src/Makefile.am @@ -52,6 +52,7 @@ nobase_include_HEADERS = \ google/protobuf/descriptor_database.h \ google/protobuf/dynamic_message.h \ google/protobuf/extension_set.h \ + google/protobuf/generated_enum_reflection.h \ google/protobuf/generated_message_util.h \ google/protobuf/generated_message_reflection.h \ google/protobuf/message.h \ @@ -91,7 +92,11 @@ libprotobuf_lite_la_SOURCES = \ google/protobuf/stubs/once.cc \ google/protobuf/stubs/hash.h \ google/protobuf/stubs/map-util.h \ - google/protobuf/stubs/stl_util-inl.h \ + google/protobuf/stubs/stl_util.h \ + google/protobuf/stubs/stringprintf.cc \ + google/protobuf/stubs/stringprintf.h \ + google/protobuf/stubs/template_util.h \ + google/protobuf/stubs/type_traits.h \ google/protobuf/extension_set.cc \ google/protobuf/generated_message_util.cc \ google/protobuf/message_lite.cc \ @@ -158,6 +163,7 @@ libprotoc_la_SOURCES = \ google/protobuf/compiler/cpp/cpp_message.h \ google/protobuf/compiler/cpp/cpp_message_field.cc \ google/protobuf/compiler/cpp/cpp_message_field.h \ + google/protobuf/compiler/cpp/cpp_options.h \ google/protobuf/compiler/cpp/cpp_primitive_field.cc \ google/protobuf/compiler/cpp/cpp_primitive_field.h \ google/protobuf/compiler/cpp/cpp_service.cc \ @@ -187,6 +193,8 @@ libprotoc_la_SOURCES = \ google/protobuf/compiler/java/java_service.h \ google/protobuf/compiler/java/java_string_field.cc \ google/protobuf/compiler/java/java_string_field.h \ + google/protobuf/compiler/java/java_doc_comment.cc \ + google/protobuf/compiler/java/java_doc_comment.h \ google/protobuf/compiler/python/python_generator.cc bin_PROGRAMS = protoc @@ -199,12 +207,14 @@ protoc_inputs = \ google/protobuf/unittest.proto \ google/protobuf/unittest_empty.proto \ google/protobuf/unittest_import.proto \ + google/protobuf/unittest_import_public.proto \ google/protobuf/unittest_mset.proto \ google/protobuf/unittest_optimize_for.proto \ google/protobuf/unittest_embed_optimize_for.proto \ google/protobuf/unittest_custom_options.proto \ google/protobuf/unittest_lite.proto \ google/protobuf/unittest_import_lite.proto \ + google/protobuf/unittest_import_public_lite.proto \ google/protobuf/unittest_lite_imports_nonlite.proto \ google/protobuf/unittest_no_generic_services.proto \ google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto @@ -228,7 +238,9 @@ protoc_lite_outputs = \ google/protobuf/unittest_lite.pb.cc \ google/protobuf/unittest_lite.pb.h \ google/protobuf/unittest_import_lite.pb.cc \ - google/protobuf/unittest_import_lite.pb.h + google/protobuf/unittest_import_lite.pb.h \ + google/protobuf/unittest_import_public_lite.pb.cc \ + google/protobuf/unittest_import_public_lite.pb.h protoc_outputs = \ $(protoc_lite_outputs) \ @@ -238,6 +250,8 @@ protoc_outputs = \ google/protobuf/unittest_empty.pb.h \ google/protobuf/unittest_import.pb.cc \ google/protobuf/unittest_import.pb.h \ + google/protobuf/unittest_import_public.pb.cc \ + google/protobuf/unittest_import_public.pb.h \ google/protobuf/unittest_mset.pb.cc \ google/protobuf/unittest_mset.pb.h \ google/protobuf/unittest_optimize_for.pb.cc \ @@ -298,6 +312,9 @@ protobuf_test_SOURCES = \ google/protobuf/stubs/once_unittest.cc \ google/protobuf/stubs/strutil_unittest.cc \ google/protobuf/stubs/structurally_valid_unittest.cc \ + google/protobuf/stubs/stringprintf_unittest.cc \ + google/protobuf/stubs/template_util_unittest.cc \ + google/protobuf/stubs/type_traits_unittest.cc \ google/protobuf/descriptor_database_unittest.cc \ google/protobuf/descriptor_unittest.cc \ google/protobuf/dynamic_message_unittest.cc \ @@ -306,6 +323,7 @@ protobuf_test_SOURCES = \ google/protobuf/message_unittest.cc \ google/protobuf/reflection_ops_unittest.cc \ google/protobuf/repeated_field_unittest.cc \ + google/protobuf/repeated_field_reflection_unittest.cc \ google/protobuf/text_format_unittest.cc \ google/protobuf/unknown_field_set_unittest.cc \ google/protobuf/wire_format_unittest.cc \ @@ -319,9 +337,11 @@ protobuf_test_SOURCES = \ google/protobuf/compiler/mock_code_generator.h \ google/protobuf/compiler/parser_unittest.cc \ google/protobuf/compiler/cpp/cpp_bootstrap_unittest.cc \ + google/protobuf/compiler/cpp/cpp_unittest.h \ google/protobuf/compiler/cpp/cpp_unittest.cc \ google/protobuf/compiler/cpp/cpp_plugin_unittest.cc \ google/protobuf/compiler/java/java_plugin_unittest.cc \ + google/protobuf/compiler/java/java_doc_comment_unittest.cc \ google/protobuf/compiler/python/python_plugin_unittest.cc \ $(COMMON_TEST_SOURCES) nodist_protobuf_test_SOURCES = $(protoc_outputs) diff --git a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.cc b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.cc index c5be9b4..b9293c9 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.cc @@ -59,12 +59,13 @@ #include <google/protobuf/descriptor.h> #include <google/protobuf/text_format.h> #include <google/protobuf/dynamic_message.h> +#include <google/protobuf/io/coded_stream.h> #include <google/protobuf/io/zero_copy_stream_impl.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> #include <google/protobuf/stubs/map-util.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { @@ -145,7 +146,7 @@ void AddTrailingSlash(string* path) { bool VerifyDirectoryExists(const string& path) { if (path.empty()) return true; - if (access(path.c_str(), W_OK) == -1) { + if (access(path.c_str(), F_OK) == -1) { cerr << path << ": " << strerror(errno) << endl; return false; } else { @@ -566,6 +567,7 @@ CommandLineInterface::CommandLineInterface() : mode_(MODE_COMPILE), error_format_(ERROR_FORMAT_GCC), imports_in_descriptor_set_(false), + source_info_in_descriptor_set_(false), disallow_services_(false), inputs_are_proto_path_relative_(false) {} CommandLineInterface::~CommandLineInterface() {} @@ -574,9 +576,23 @@ void CommandLineInterface::RegisterGenerator(const string& flag_name, CodeGenerator* generator, const string& help_text) { GeneratorInfo info; + info.flag_name = flag_name; info.generator = generator; info.help_text = help_text; - generators_[flag_name] = info; + generators_by_flag_name_[flag_name] = info; +} + +void CommandLineInterface::RegisterGenerator(const string& flag_name, + const string& option_flag_name, + CodeGenerator* generator, + const string& help_text) { + GeneratorInfo info; + info.flag_name = flag_name; + info.option_flag_name = option_flag_name; + info.generator = generator; + info.help_text = help_text; + generators_by_flag_name_[flag_name] = info; + generators_by_option_name_[option_flag_name] = info; } void CommandLineInterface::AllowPlugins(const string& exe_name_prefix) { @@ -585,7 +601,14 @@ void CommandLineInterface::AllowPlugins(const string& exe_name_prefix) { int CommandLineInterface::Run(int argc, const char* const argv[]) { Clear(); - if (!ParseArguments(argc, argv)) return 1; + switch (ParseArguments(argc, argv)) { + case PARSE_ARGUMENT_DONE_AND_EXIT: + return 0; + case PARSE_ARGUMENT_FAIL: + return 1; + case PARSE_ARGUMENT_DONE_AND_CONTINUE: + break; + } // Set up the source tree. DiskSourceTree source_tree; @@ -713,6 +736,7 @@ void CommandLineInterface::Clear() { mode_ = MODE_COMPILE; imports_in_descriptor_set_ = false; + source_info_in_descriptor_set_ = false; disallow_services_ = false; } @@ -755,7 +779,8 @@ bool CommandLineInterface::MakeInputsBeProtoPathRelative( return true; } -bool CommandLineInterface::ParseArguments(int argc, const char* const argv[]) { +CommandLineInterface::ParseArgumentStatus +CommandLineInterface::ParseArguments(int argc, const char* const argv[]) { executable_name_ = argv[0]; // Iterate through all arguments and parse them. @@ -769,41 +794,50 @@ bool CommandLineInterface::ParseArguments(int argc, const char* const argv[]) { if (name == "--decode") { cerr << "To decode an unknown message, use --decode_raw." << endl; } - return false; + return PARSE_ARGUMENT_FAIL; } else { ++i; value = argv[i]; } } - if (!InterpretArgument(name, value)) return false; + ParseArgumentStatus status = InterpretArgument(name, value); + if (status != PARSE_ARGUMENT_DONE_AND_CONTINUE) + return status; } // If no --proto_path was given, use the current working directory. if (proto_path_.empty()) { - proto_path_.push_back(make_pair<string, string>("", ".")); + // Don't use make_pair as the old/default standard library on Solaris + // doesn't support it without explicit template parameters, which are + // incompatible with C++0x's make_pair. + proto_path_.push_back(pair<string, string>("", ".")); } // Check some errror cases. bool decoding_raw = (mode_ == MODE_DECODE) && codec_type_.empty(); if (decoding_raw && !input_files_.empty()) { cerr << "When using --decode_raw, no input files should be given." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } else if (!decoding_raw && input_files_.empty()) { cerr << "Missing input file." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } if (mode_ == MODE_COMPILE && output_directives_.empty() && descriptor_set_name_.empty()) { cerr << "Missing output directives." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } if (imports_in_descriptor_set_ && descriptor_set_name_.empty()) { cerr << "--include_imports only makes sense when combined with " "--descriptor_set_out." << endl; } + if (source_info_in_descriptor_set_ && descriptor_set_name_.empty()) { + cerr << "--include_source_info only makes sense when combined with " + "--descriptor_set_out." << endl; + } - return true; + return PARSE_ARGUMENT_DONE_AND_CONTINUE; } bool CommandLineInterface::ParseArgument(const char* arg, @@ -853,6 +887,7 @@ bool CommandLineInterface::ParseArgument(const char* arg, if (*name == "-h" || *name == "--help" || *name == "--disallow_services" || *name == "--include_imports" || + *name == "--include_source_info" || *name == "--version" || *name == "--decode_raw") { // HACK: These are the only flags that don't take a value. @@ -865,8 +900,9 @@ bool CommandLineInterface::ParseArgument(const char* arg, return true; } -bool CommandLineInterface::InterpretArgument(const string& name, - const string& value) { +CommandLineInterface::ParseArgumentStatus +CommandLineInterface::InterpretArgument(const string& name, + const string& value) { if (name.empty()) { // Not a flag. Just a filename. if (value.empty()) { @@ -874,7 +910,7 @@ bool CommandLineInterface::InterpretArgument(const string& name, "arguments to " << executable_name_ << ". This is actually " "sort of hard to do. Congrats. Unfortunately it is not valid " "input so the program is going to die now." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } input_files_.push_back(value); @@ -902,7 +938,7 @@ bool CommandLineInterface::InterpretArgument(const string& name, if (disk_path.empty()) { cerr << "--proto_path passed empty directory name. (Use \".\" for " "current directory.)" << endl; - return false; + return PARSE_ARGUMENT_FAIL; } // Make sure disk path exists, warn otherwise. @@ -910,35 +946,45 @@ bool CommandLineInterface::InterpretArgument(const string& name, cerr << disk_path << ": warning: directory does not exist." << endl; } - proto_path_.push_back(make_pair(virtual_path, disk_path)); + // Don't use make_pair as the old/default standard library on Solaris + // doesn't support it without explicit template parameters, which are + // incompatible with C++0x's make_pair. + proto_path_.push_back(pair<string, string>(virtual_path, disk_path)); } } else if (name == "-o" || name == "--descriptor_set_out") { if (!descriptor_set_name_.empty()) { cerr << name << " may only be passed once." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } if (value.empty()) { cerr << name << " requires a non-empty value." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } if (mode_ != MODE_COMPILE) { cerr << "Cannot use --encode or --decode and generate descriptors at the " "same time." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } descriptor_set_name_ = value; } else if (name == "--include_imports") { if (imports_in_descriptor_set_) { cerr << name << " may only be passed once." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } imports_in_descriptor_set_ = true; + } else if (name == "--include_source_info") { + if (source_info_in_descriptor_set_) { + cerr << name << " may only be passed once." << endl; + return PARSE_ARGUMENT_FAIL; + } + source_info_in_descriptor_set_ = true; + } else if (name == "-h" || name == "--help") { PrintHelpText(); - return false; // Exit without running compiler. + return PARSE_ARGUMENT_DONE_AND_EXIT; // Exit without running compiler. } else if (name == "--version") { if (!version_info_.empty()) { @@ -947,7 +993,7 @@ bool CommandLineInterface::InterpretArgument(const string& name, cout << "libprotoc " << protobuf::internal::VersionString(GOOGLE_PROTOBUF_VERSION) << endl; - return false; // Exit without running compiler. + return PARSE_ARGUMENT_DONE_AND_EXIT; // Exit without running compiler. } else if (name == "--disallow_services") { disallow_services_ = true; @@ -956,12 +1002,12 @@ bool CommandLineInterface::InterpretArgument(const string& name, name == "--decode_raw") { if (mode_ != MODE_COMPILE) { cerr << "Only one of --encode and --decode can be specified." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } if (!output_directives_.empty() || !descriptor_set_name_.empty()) { cerr << "Cannot use " << name << " and generate code or descriptors at the same time." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } mode_ = (name == "--encode") ? MODE_ENCODE : MODE_DECODE; @@ -971,10 +1017,10 @@ bool CommandLineInterface::InterpretArgument(const string& name, if (name == "--decode") { cerr << "To decode an unknown message, use --decode_raw." << endl; } - return false; + return PARSE_ARGUMENT_FAIL; } else if (!value.empty() && name == "--decode_raw") { cerr << "--decode_raw does not take a parameter." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } codec_type_ = value; @@ -986,16 +1032,16 @@ bool CommandLineInterface::InterpretArgument(const string& name, error_format_ = ERROR_FORMAT_MSVS; } else { cerr << "Unknown error format: " << value << endl; - return false; + return PARSE_ARGUMENT_FAIL; } } else if (name == "--plugin") { if (plugin_prefix_.empty()) { cerr << "This compiler does not support plugins." << endl; - return false; + return PARSE_ARGUMENT_FAIL; } - string name; + string plugin_name; string path; string::size_type equals_pos = value.find_first_of('='); @@ -1003,57 +1049,68 @@ bool CommandLineInterface::InterpretArgument(const string& name, // Use the basename of the file. string::size_type slash_pos = value.find_last_of('/'); if (slash_pos == string::npos) { - name = value; + plugin_name = value; } else { - name = value.substr(slash_pos + 1); + plugin_name = value.substr(slash_pos + 1); } path = value; } else { - name = value.substr(0, equals_pos); + plugin_name = value.substr(0, equals_pos); path = value.substr(equals_pos + 1); } - plugins_[name] = path; + plugins_[plugin_name] = path; } else { // Some other flag. Look it up in the generators list. - const GeneratorInfo* generator_info = FindOrNull(generators_, name); + const GeneratorInfo* generator_info = + FindOrNull(generators_by_flag_name_, name); if (generator_info == NULL && (plugin_prefix_.empty() || !HasSuffixString(name, "_out"))) { - cerr << "Unknown flag: " << name << endl; - return false; - } + // Check if it's a generator option flag. + generator_info = FindOrNull(generators_by_option_name_, name); + if (generator_info == NULL) { + cerr << "Unknown flag: " << name << endl; + return PARSE_ARGUMENT_FAIL; + } else { + string* parameters = &generator_parameters_[generator_info->flag_name]; + if (!parameters->empty()) { + parameters->append(","); + } + parameters->append(value); + } + } else { + // It's an output flag. Add it to the output directives. + if (mode_ != MODE_COMPILE) { + cerr << "Cannot use --encode or --decode and generate code at the " + "same time." << endl; + return PARSE_ARGUMENT_FAIL; + } - // It's an output flag. Add it to the output directives. - if (mode_ != MODE_COMPILE) { - cerr << "Cannot use --encode or --decode and generate code at the " - "same time." << endl; - return false; - } + OutputDirective directive; + directive.name = name; + if (generator_info == NULL) { + directive.generator = NULL; + } else { + directive.generator = generator_info->generator; + } - OutputDirective directive; - directive.name = name; - if (generator_info == NULL) { - directive.generator = NULL; - } else { - directive.generator = generator_info->generator; - } + // Split value at ':' to separate the generator parameter from the + // filename. However, avoid doing this if the colon is part of a valid + // Windows-style absolute path. + string::size_type colon_pos = value.find_first_of(':'); + if (colon_pos == string::npos || IsWindowsAbsolutePath(value)) { + directive.output_location = value; + } else { + directive.parameter = value.substr(0, colon_pos); + directive.output_location = value.substr(colon_pos + 1); + } - // Split value at ':' to separate the generator parameter from the - // filename. However, avoid doing this if the colon is part of a valid - // Windows-style absolute path. - string::size_type colon_pos = value.find_first_of(':'); - if (colon_pos == string::npos || IsWindowsAbsolutePath(value)) { - directive.output_location = value; - } else { - directive.parameter = value.substr(0, colon_pos); - directive.output_location = value.substr(colon_pos + 1); + output_directives_.push_back(directive); } - - output_directives_.push_back(directive); } - return true; + return PARSE_ARGUMENT_DONE_AND_CONTINUE; } void CommandLineInterface::PrintHelpText() { @@ -1086,6 +1143,12 @@ void CommandLineInterface::PrintHelpText() { " --include_imports When using --descriptor_set_out, also include\n" " all dependencies of the input files in the\n" " set, so that the set is self-contained.\n" +" --include_source_info When using --descriptor_set_out, do not strip\n" +" SourceCodeInfo from the FileDescriptorProto.\n" +" This results in vastly larger descriptors that\n" +" include information about the original\n" +" location of each decl in the source file as\n" +" well as surrounding comments.\n" " --error_format=FORMAT Set the format in which to print errors.\n" " FORMAT may be 'gcc' (the default) or 'msvs'\n" " (Microsoft Visual Studio format)." << endl; @@ -1101,8 +1164,8 @@ void CommandLineInterface::PrintHelpText() { " the executable's own name differs." << endl; } - for (GeneratorMap::iterator iter = generators_.begin(); - iter != generators_.end(); ++iter) { + for (GeneratorMap::iterator iter = generators_by_flag_name_.begin(); + iter != generators_by_flag_name_.end(); ++iter) { // FIXME(kenton): If the text is long enough it will wrap, which is ugly, // but fixing this nicely (e.g. splitting on spaces) is probably more // trouble than it's worth. @@ -1136,10 +1199,16 @@ bool CommandLineInterface::GenerateOutput( } } else { // Regular generator. + string parameters = output_directive.parameter; + if (!generator_parameters_[output_directive.name].empty()) { + if (!parameters.empty()) { + parameters.append(","); + } + parameters.append(generator_parameters_[output_directive.name]); + } for (int i = 0; i < parsed_files.size(); i++) { - if (!output_directive.generator->Generate( - parsed_files[i], output_directive.parameter, - generator_context, &error)) { + if (!output_directive.generator->Generate(parsed_files[i], parameters, + generator_context, &error)) { // Generator returned an error. cerr << output_directive.name << ": " << parsed_files[i]->name() << ": " << error << endl; @@ -1168,8 +1237,9 @@ bool CommandLineInterface::GeneratePluginOutput( set<const FileDescriptor*> already_seen; for (int i = 0; i < parsed_files.size(); i++) { request.add_file_to_generate(parsed_files[i]->name()); - GetTransitiveDependencies(parsed_files[i], &already_seen, - request.mutable_proto_file()); + GetTransitiveDependencies(parsed_files[i], + true, // Include source code info. + &already_seen, request.mutable_proto_file()); } // Invoke the plugin. @@ -1299,12 +1369,17 @@ bool CommandLineInterface::WriteDescriptorSet( if (imports_in_descriptor_set_) { set<const FileDescriptor*> already_seen; for (int i = 0; i < parsed_files.size(); i++) { - GetTransitiveDependencies( - parsed_files[i], &already_seen, file_set.mutable_file()); + GetTransitiveDependencies(parsed_files[i], + source_info_in_descriptor_set_, + &already_seen, file_set.mutable_file()); } } else { for (int i = 0; i < parsed_files.size(); i++) { - parsed_files[i]->CopyTo(file_set.add_file()); + FileDescriptorProto* file_proto = file_set.add_file(); + parsed_files[i]->CopyTo(file_proto); + if (source_info_in_descriptor_set_) { + parsed_files[i]->CopySourceCodeInfoTo(file_proto); + } } } @@ -1334,7 +1409,7 @@ bool CommandLineInterface::WriteDescriptorSet( } void CommandLineInterface::GetTransitiveDependencies( - const FileDescriptor* file, + const FileDescriptor* file, bool include_source_code_info, set<const FileDescriptor*>* already_seen, RepeatedPtrField<FileDescriptorProto>* output) { if (!already_seen->insert(file).second) { @@ -1344,11 +1419,16 @@ void CommandLineInterface::GetTransitiveDependencies( // Add all dependencies. for (int i = 0; i < file->dependency_count(); i++) { - GetTransitiveDependencies(file->dependency(i), already_seen, output); + GetTransitiveDependencies(file->dependency(i), include_source_code_info, + already_seen, output); } // Add this file. - file->CopyTo(output->Add()); + FileDescriptorProto* new_descriptor = output->Add(); + file->CopyTo(new_descriptor); + if (include_source_code_info) { + file->CopySourceCodeInfoTo(new_descriptor); + } } diff --git a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.h b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.h index 0b507d8..86ea9bd 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.h +++ b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface.h @@ -112,6 +112,19 @@ class LIBPROTOC_EXPORT CommandLineInterface { CodeGenerator* generator, const string& help_text); + // Register a code generator for a language. + // Besides flag_name you can specify another option_flag_name that could be + // used to pass extra parameters to the registered code generator. + // Suppose you have registered a generator by calling: + // command_line_interface.RegisterGenerator("--foo_out", "--foo_opt", ...) + // Then you could invoke the compiler with a command like: + // protoc --foo_out=enable_bar:outdir --foo_opt=enable_baz + // This will pass "enable_bar,enable_baz" as the parameter to the generator. + void RegisterGenerator(const string& flag_name, + const string& option_flag_name, + CodeGenerator* generator, + const string& help_text); + // Enables "plugins". In this mode, if a command-line flag ends with "_out" // but does not match any registered generator, the compiler will attempt to // find a "plugin" to implement the generator. Plugins are just executables. @@ -186,8 +199,15 @@ class LIBPROTOC_EXPORT CommandLineInterface { bool MakeInputsBeProtoPathRelative( DiskSourceTree* source_tree); + // Return status for ParseArguments() and InterpretArgument(). + enum ParseArgumentStatus { + PARSE_ARGUMENT_DONE_AND_CONTINUE, + PARSE_ARGUMENT_DONE_AND_EXIT, + PARSE_ARGUMENT_FAIL + }; + // Parse all command-line arguments. - bool ParseArguments(int argc, const char* const argv[]); + ParseArgumentStatus ParseArguments(int argc, const char* const argv[]); // Parses a command-line argument into a name/value pair. Returns // true if the next argument in the argv should be used as the value, @@ -203,7 +223,8 @@ class LIBPROTOC_EXPORT CommandLineInterface { bool ParseArgument(const char* arg, string* name, string* value); // Interprets arguments parsed with ParseArgument. - bool InterpretArgument(const string& name, const string& value); + ParseArgumentStatus InterpretArgument(const string& name, + const string& value); // Print the --help text to stderr. void PrintHelpText(); @@ -230,9 +251,11 @@ class LIBPROTOC_EXPORT CommandLineInterface { // protos will be ordered such that every file is listed before any file that // depends on it, so that you can call DescriptorPool::BuildFile() on them // in order. Any files in *already_seen will not be added, and each file - // added will be inserted into *already_seen. + // added will be inserted into *already_seen. If include_source_code_info is + // true then include the source code information in the FileDescriptorProtos. static void GetTransitiveDependencies( const FileDescriptor* file, + bool include_source_code_info, set<const FileDescriptor*>* already_seen, RepeatedPtrField<FileDescriptorProto>* output); @@ -244,13 +267,21 @@ class LIBPROTOC_EXPORT CommandLineInterface { // Version info set with SetVersionInfo(). string version_info_; - // Map from flag names to registered generators. + // Registered generators. struct GeneratorInfo { + string flag_name; + string option_flag_name; CodeGenerator* generator; string help_text; }; typedef map<string, GeneratorInfo> GeneratorMap; - GeneratorMap generators_; + GeneratorMap generators_by_flag_name_; + GeneratorMap generators_by_option_name_; + // A map from generator names to the parameters specified using the option + // flag. For example, if the user invokes the compiler with: + // protoc --foo_out=outputdir --foo_opt=enable_bar ... + // Then there will be an entry ("--foo_out", "enable_bar") in this map. + map<string, string> generator_parameters_; // See AllowPlugins(). If this is empty, plugins aren't allowed. string plugin_prefix_; @@ -302,6 +333,10 @@ class LIBPROTOC_EXPORT CommandLineInterface { // the .proto files listed on the command-line are added. bool imports_in_descriptor_set_; + // True if --include_source_info was given, meaning that we should not strip + // SourceCodeInfo from the DescriptorSet. + bool source_info_in_descriptor_set_; + // Was the --disallow_services flag used? bool disallow_services_; diff --git a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface_unittest.cc b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface_unittest.cc index d5b3a1d..1655992 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/command_line_interface_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/command_line_interface_unittest.cc @@ -122,6 +122,10 @@ class CommandLineInterfaceTest : public testing::Test { // substring. void ExpectErrorSubstring(const string& expected_substring); + // Like ExpectErrorSubstring, but checks that Run() returned zero. + void ExpectErrorSubstringWithZeroReturnCode( + const string& expected_substring); + // Returns true if ExpectErrorSubstring(expected_substring) would pass, but // does not fail otherwise. bool HasAlternateErrorSubstring(const string& expected_substring); @@ -225,7 +229,7 @@ void CommandLineInterfaceTest::SetUp() { // Register generators. CodeGenerator* generator = new MockCodeGenerator("test_generator"); mock_generators_to_delete_.push_back(generator); - cli_.RegisterGenerator("--test_out", generator, "Test output."); + cli_.RegisterGenerator("--test_out", "--test_opt", generator, "Test output."); cli_.RegisterGenerator("-t", generator, "Test output."); generator = new MockCodeGenerator("alt_generator"); @@ -345,6 +349,12 @@ void CommandLineInterfaceTest::ExpectErrorSubstring( EXPECT_PRED_FORMAT2(testing::IsSubstring, expected_substring, error_text_); } +void CommandLineInterfaceTest::ExpectErrorSubstringWithZeroReturnCode( + const string& expected_substring) { + EXPECT_EQ(0, return_code_); + EXPECT_PRED_FORMAT2(testing::IsSubstring, expected_substring, error_text_); +} + bool CommandLineInterfaceTest::HasAlternateErrorSubstring( const string& expected_substring) { EXPECT_NE(0, return_code_); @@ -544,6 +554,32 @@ TEST_F(CommandLineInterfaceTest, GeneratorParameters) { ExpectGenerated("test_plugin", "TestPluginParameter", "foo.proto", "Foo"); } +TEST_F(CommandLineInterfaceTest, ExtraGeneratorParameters) { + // Test that generator parameters specified with the option flag are + // correctly passed to the code generator. + + CreateTempFile("foo.proto", + "syntax = \"proto2\";\n" + "message Foo {}\n"); + // Create the "a" and "b" sub-directories. + CreateTempDir("a"); + CreateTempDir("b"); + + Run("protocol_compiler " + "--test_opt=foo1 " + "--test_out=bar:$tmpdir/a " + "--test_opt=foo2 " + "--test_out=baz:$tmpdir/b " + "--test_opt=foo3 " + "--proto_path=$tmpdir foo.proto"); + + ExpectNoErrors(); + ExpectGenerated( + "test_generator", "bar,foo1,foo2,foo3", "foo.proto", "Foo", "a"); + ExpectGenerated( + "test_generator", "baz,foo1,foo2,foo3", "foo.proto", "Foo", "b"); +} + TEST_F(CommandLineInterfaceTest, Insert) { // Test running a generator that inserts code into another's output. @@ -779,6 +815,33 @@ TEST_F(CommandLineInterfaceTest, WriteDescriptorSet) { if (HasFatalFailure()) return; ASSERT_EQ(1, descriptor_set.file_size()); EXPECT_EQ("bar.proto", descriptor_set.file(0).name()); + // Descriptor set should not have source code info. + EXPECT_FALSE(descriptor_set.file(0).has_source_code_info()); +} + +TEST_F(CommandLineInterfaceTest, WriteDescriptorSetWithSourceInfo) { + CreateTempFile("foo.proto", + "syntax = \"proto2\";\n" + "message Foo {}\n"); + CreateTempFile("bar.proto", + "syntax = \"proto2\";\n" + "import \"foo.proto\";\n" + "message Bar {\n" + " optional Foo foo = 1;\n" + "}\n"); + + Run("protocol_compiler --descriptor_set_out=$tmpdir/descriptor_set " + "--include_source_info --proto_path=$tmpdir bar.proto"); + + ExpectNoErrors(); + + FileDescriptorSet descriptor_set; + ReadDescriptorSet("descriptor_set", &descriptor_set); + if (HasFatalFailure()) return; + ASSERT_EQ(1, descriptor_set.file_size()); + EXPECT_EQ("bar.proto", descriptor_set.file(0).name()); + // Source code info included. + EXPECT_TRUE(descriptor_set.file(0).has_source_code_info()); } TEST_F(CommandLineInterfaceTest, WriteTransitiveDescriptorSet) { @@ -807,6 +870,40 @@ TEST_F(CommandLineInterfaceTest, WriteTransitiveDescriptorSet) { } EXPECT_EQ("foo.proto", descriptor_set.file(0).name()); EXPECT_EQ("bar.proto", descriptor_set.file(1).name()); + // Descriptor set should not have source code info. + EXPECT_FALSE(descriptor_set.file(0).has_source_code_info()); + EXPECT_FALSE(descriptor_set.file(1).has_source_code_info()); +} + +TEST_F(CommandLineInterfaceTest, WriteTransitiveDescriptorSetWithSourceInfo) { + CreateTempFile("foo.proto", + "syntax = \"proto2\";\n" + "message Foo {}\n"); + CreateTempFile("bar.proto", + "syntax = \"proto2\";\n" + "import \"foo.proto\";\n" + "message Bar {\n" + " optional Foo foo = 1;\n" + "}\n"); + + Run("protocol_compiler --descriptor_set_out=$tmpdir/descriptor_set " + "--include_imports --include_source_info --proto_path=$tmpdir bar.proto"); + + ExpectNoErrors(); + + FileDescriptorSet descriptor_set; + ReadDescriptorSet("descriptor_set", &descriptor_set); + if (HasFatalFailure()) return; + ASSERT_EQ(2, descriptor_set.file_size()); + if (descriptor_set.file(0).name() == "bar.proto") { + std::swap(descriptor_set.mutable_file()->mutable_data()[0], + descriptor_set.mutable_file()->mutable_data()[1]); + } + EXPECT_EQ("foo.proto", descriptor_set.file(0).name()); + EXPECT_EQ("bar.proto", descriptor_set.file(1).name()); + // Source code info included. + EXPECT_TRUE(descriptor_set.file(0).has_source_code_info()); + EXPECT_TRUE(descriptor_set.file(1).has_source_code_info()); } // ------------------------------------------------------------------- @@ -1129,6 +1226,17 @@ TEST_F(CommandLineInterfaceTest, GeneratorPluginCrash) { #endif } +TEST_F(CommandLineInterfaceTest, PluginReceivesSourceCodeInfo) { + CreateTempFile("foo.proto", + "syntax = \"proto2\";\n" + "message MockCodeGenerator_HasSourceCodeInfo {}\n"); + + Run("protocol_compiler --plug_out=$tmpdir --proto_path=$tmpdir foo.proto"); + + ExpectErrorSubstring( + "Saw message type MockCodeGenerator_HasSourceCodeInfo: 1."); +} + TEST_F(CommandLineInterfaceTest, GeneratorPluginNotFound) { // Test what happens if the plugin isn't found. @@ -1171,11 +1279,11 @@ TEST_F(CommandLineInterfaceTest, GeneratorPluginNotAllowed) { TEST_F(CommandLineInterfaceTest, HelpText) { Run("test_exec_name --help"); - ExpectErrorSubstring("Usage: test_exec_name "); - ExpectErrorSubstring("--test_out=OUT_DIR"); - ExpectErrorSubstring("Test output."); - ExpectErrorSubstring("--alt_out=OUT_DIR"); - ExpectErrorSubstring("Alt output."); + ExpectErrorSubstringWithZeroReturnCode("Usage: test_exec_name "); + ExpectErrorSubstringWithZeroReturnCode("--test_out=OUT_DIR"); + ExpectErrorSubstringWithZeroReturnCode("Test output."); + ExpectErrorSubstringWithZeroReturnCode("--alt_out=OUT_DIR"); + ExpectErrorSubstringWithZeroReturnCode("Alt output."); } TEST_F(CommandLineInterfaceTest, GccFormatErrors) { diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_bootstrap_unittest.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_bootstrap_unittest.cc index bcfa502..b7c1766 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_bootstrap_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_bootstrap_unittest.cc @@ -48,8 +48,8 @@ #include <google/protobuf/compiler/importer.h> #include <google/protobuf/descriptor.h> #include <google/protobuf/io/zero_copy_stream_impl.h> -#include <google/protobuf/stubs/stl_util-inl.h> #include <google/protobuf/stubs/map-util.h> +#include <google/protobuf/stubs/stl_util.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.cc index 76d2b79..67c12d7 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.cc @@ -46,10 +46,10 @@ namespace compiler { namespace cpp { EnumGenerator::EnumGenerator(const EnumDescriptor* descriptor, - const string& dllexport_decl) + const Options& options) : descriptor_(descriptor), classname_(ClassName(descriptor, false)), - dllexport_decl_(dllexport_decl) { + options_(options) { } EnumGenerator::~EnumGenerator() {} @@ -88,10 +88,10 @@ void EnumGenerator::GenerateDefinition(io::Printer* printer) { vars["min_name"] = min_value->name(); vars["max_name"] = max_value->name(); - if (dllexport_decl_.empty()) { + if (options_.dllexport_decl.empty()) { vars["dllexport"] = ""; } else { - vars["dllexport"] = dllexport_decl_ + " "; + vars["dllexport"] = options_.dllexport_decl + " "; } printer->Print(vars, diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.h index 58f7721..2e85a0b 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum.h @@ -36,8 +36,10 @@ #define GOOGLE_PROTOBUF_COMPILER_CPP_ENUM_H__ #include <string> +#include <google/protobuf/compiler/cpp/cpp_options.h> #include <google/protobuf/descriptor.h> + namespace google { namespace protobuf { namespace io { @@ -53,7 +55,7 @@ class EnumGenerator { public: // See generator.cc for the meaning of dllexport_decl. explicit EnumGenerator(const EnumDescriptor* descriptor, - const string& dllexport_decl); + const Options& options); ~EnumGenerator(); // Header stuff. @@ -86,7 +88,7 @@ class EnumGenerator { private: const EnumDescriptor* descriptor_; string classname_; - string dllexport_decl_; + Options options_; GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(EnumGenerator); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.cc index a369f41..6e1620d 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.cc @@ -46,8 +46,9 @@ namespace cpp { namespace { void SetEnumVariables(const FieldDescriptor* descriptor, - map<string, string>* variables) { - SetCommonFieldVariables(descriptor, variables); + map<string, string>* variables, + const Options& options) { + SetCommonFieldVariables(descriptor, variables, options); const EnumValueDescriptor* default_value = descriptor->default_value_enum(); (*variables)["type"] = ClassName(descriptor->enum_type(), true); (*variables)["default"] = SimpleItoa(default_value->number()); @@ -58,9 +59,10 @@ void SetEnumVariables(const FieldDescriptor* descriptor, // =================================================================== EnumFieldGenerator:: -EnumFieldGenerator(const FieldDescriptor* descriptor) +EnumFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetEnumVariables(descriptor, &variables_); + SetEnumVariables(descriptor, &variables_, options); } EnumFieldGenerator::~EnumFieldGenerator() {} @@ -84,7 +86,7 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { " return static_cast< $type$ >($name$_);\n" "}\n" "inline void $classname$::set_$name$($type$ value) {\n" - " GOOGLE_DCHECK($type$_IsValid(value));\n" + " assert($type$_IsValid(value));\n" " set_has_$name$();\n" " $name$_ = value;\n" "}\n"); @@ -152,9 +154,10 @@ GenerateByteSize(io::Printer* printer) const { // =================================================================== RepeatedEnumFieldGenerator:: -RepeatedEnumFieldGenerator(const FieldDescriptor* descriptor) +RepeatedEnumFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetEnumVariables(descriptor, &variables_); + SetEnumVariables(descriptor, &variables_, options); } RepeatedEnumFieldGenerator::~RepeatedEnumFieldGenerator() {} @@ -187,11 +190,11 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { " return static_cast< $type$ >($name$_.Get(index));\n" "}\n" "inline void $classname$::set_$name$(int index, $type$ value) {\n" - " GOOGLE_DCHECK($type$_IsValid(value));\n" + " assert($type$_IsValid(value));\n" " $name$_.Set(index, value);\n" "}\n" "inline void $classname$::add_$name$($type$ value) {\n" - " GOOGLE_DCHECK($type$_IsValid(value));\n" + " assert($type$_IsValid(value));\n" " $name$_.Add(value);\n" "}\n"); printer->Print(variables_, @@ -345,7 +348,9 @@ GenerateByteSize(io::Printer* printer) const { " total_size += $tag_size$ +\n" " ::google::protobuf::internal::WireFormatLite::Int32Size(data_size);\n" "}\n" + "GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();\n" "_$name$_cached_byte_size_ = data_size;\n" + "GOOGLE_SAFE_CONCURRENT_WRITES_END();\n" "total_size += data_size;\n"); } else { printer->Print(variables_, diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.h index 0793430..6577008 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_enum_field.h @@ -46,7 +46,8 @@ namespace cpp { class EnumFieldGenerator : public FieldGenerator { public: - explicit EnumFieldGenerator(const FieldDescriptor* descriptor); + explicit EnumFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~EnumFieldGenerator(); // implements FieldGenerator --------------------------------------- @@ -71,7 +72,8 @@ class EnumFieldGenerator : public FieldGenerator { class RepeatedEnumFieldGenerator : public FieldGenerator { public: - explicit RepeatedEnumFieldGenerator(const FieldDescriptor* descriptor); + explicit RepeatedEnumFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~RepeatedEnumFieldGenerator(); // implements FieldGenerator --------------------------------------- diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.cc index 658a707..ef56b5e 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.cc @@ -57,9 +57,9 @@ string ExtendeeClassName(const FieldDescriptor* descriptor) { } // anonymous namespace ExtensionGenerator::ExtensionGenerator(const FieldDescriptor* descriptor, - const string& dllexport_decl) + const Options& options) : descriptor_(descriptor), - dllexport_decl_(dllexport_decl) { + options_(options) { // Construct type_traits_. if (descriptor_->is_repeated()) { type_traits_ = "Repeated"; @@ -106,8 +106,8 @@ void ExtensionGenerator::GenerateDeclaration(io::Printer* printer) { // export/import specifier. if (descriptor_->extension_scope() == NULL) { vars["qualifier"] = "extern"; - if (!dllexport_decl_.empty()) { - vars["qualifier"] = dllexport_decl_ + " " + vars["qualifier"]; + if (!options_.dllexport_decl.empty()) { + vars["qualifier"] = options_.dllexport_decl + " " + vars["qualifier"]; } } else { vars["qualifier"] = "static"; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.h index 3068b09..50ad035 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_extension.h @@ -37,6 +37,7 @@ #include <string> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/compiler/cpp/cpp_options.h> namespace google { namespace protobuf { @@ -56,8 +57,8 @@ namespace cpp { class ExtensionGenerator { public: // See generator.cc for the meaning of dllexport_decl. - explicit ExtensionGenerator(const FieldDescriptor* descriptor, - const string& dllexport_decl); + explicit ExtensionGenerator(const FieldDescriptor* desycriptor, + const Options& options); ~ExtensionGenerator(); // Header stuff. @@ -72,7 +73,7 @@ class ExtensionGenerator { private: const FieldDescriptor* descriptor_; string type_traits_; - string dllexport_decl_; + Options options_; GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(ExtensionGenerator); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.cc index 103cac4..0786176 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.cc @@ -52,7 +52,8 @@ namespace cpp { using internal::WireFormat; void SetCommonFieldVariables(const FieldDescriptor* descriptor, - map<string, string>* variables) { + map<string, string>* variables, + const Options& options) { (*variables)["name"] = FieldName(descriptor); (*variables)["index"] = SimpleItoa(descriptor->index()); (*variables)["number"] = SimpleItoa(descriptor->number()); @@ -64,6 +65,7 @@ void SetCommonFieldVariables(const FieldDescriptor* descriptor, (*variables)["deprecation"] = descriptor->options().deprecated() ? " PROTOBUF_DEPRECATED" : ""; + (*variables)["cppget"] = "Get"; } FieldGenerator::~FieldGenerator() {} @@ -80,46 +82,47 @@ GenerateMergeFromCodedStreamWithPacking(io::Printer* printer) const { } -FieldGeneratorMap::FieldGeneratorMap(const Descriptor* descriptor) +FieldGeneratorMap::FieldGeneratorMap(const Descriptor* descriptor, + const Options& options) : descriptor_(descriptor), - field_generators_( - new scoped_ptr<FieldGenerator>[descriptor->field_count()]) { + field_generators_(new scoped_ptr<FieldGenerator>[descriptor->field_count()]) { // Construct all the FieldGenerators. for (int i = 0; i < descriptor->field_count(); i++) { - field_generators_[i].reset(MakeGenerator(descriptor->field(i))); + field_generators_[i].reset(MakeGenerator(descriptor->field(i), options)); } } -FieldGenerator* FieldGeneratorMap::MakeGenerator(const FieldDescriptor* field) { +FieldGenerator* FieldGeneratorMap::MakeGenerator(const FieldDescriptor* field, + const Options& options) { if (field->is_repeated()) { switch (field->cpp_type()) { case FieldDescriptor::CPPTYPE_MESSAGE: - return new RepeatedMessageFieldGenerator(field); + return new RepeatedMessageFieldGenerator(field, options); case FieldDescriptor::CPPTYPE_STRING: switch (field->options().ctype()) { default: // RepeatedStringFieldGenerator handles unknown ctypes. case FieldOptions::STRING: - return new RepeatedStringFieldGenerator(field); + return new RepeatedStringFieldGenerator(field, options); } case FieldDescriptor::CPPTYPE_ENUM: - return new RepeatedEnumFieldGenerator(field); + return new RepeatedEnumFieldGenerator(field, options); default: - return new RepeatedPrimitiveFieldGenerator(field); + return new RepeatedPrimitiveFieldGenerator(field, options); } } else { switch (field->cpp_type()) { case FieldDescriptor::CPPTYPE_MESSAGE: - return new MessageFieldGenerator(field); + return new MessageFieldGenerator(field, options); case FieldDescriptor::CPPTYPE_STRING: switch (field->options().ctype()) { default: // StringFieldGenerator handles unknown ctypes. case FieldOptions::STRING: - return new StringFieldGenerator(field); + return new StringFieldGenerator(field, options); } case FieldDescriptor::CPPTYPE_ENUM: - return new EnumFieldGenerator(field); + return new EnumFieldGenerator(field, options); default: - return new PrimitiveFieldGenerator(field); + return new PrimitiveFieldGenerator(field, options); } } } diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.h index c303a33..f7d99b1 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_field.h @@ -40,6 +40,7 @@ #include <google/protobuf/stubs/common.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/compiler/cpp/cpp_options.h> namespace google { namespace protobuf { @@ -57,7 +58,8 @@ namespace cpp { // ['name', 'index', 'number', 'classname', 'declared_type', 'tag_size', // 'deprecation']. void SetCommonFieldVariables(const FieldDescriptor* descriptor, - map<string, string>* variables); + map<string, string>* variables, + const Options& options); class FieldGenerator { public: @@ -114,6 +116,13 @@ class FieldGenerator { // Most field types don't need this, so the default implementation is empty. virtual void GenerateDestructorCode(io::Printer* printer) const {} + // Generate code that allocates the fields's default instance. + virtual void GenerateDefaultInstanceAllocator(io::Printer* printer) const {} + + // Generate code that should be run when ShutdownProtobufLibrary() is called, + // to delete all dynamically-allocated objects. + virtual void GenerateShutdownCode(io::Printer* printer) const {} + // Generate lines to decode this field, which will be placed inside the // message's MergeFromCodedStream() method. virtual void GenerateMergeFromCodedStream(io::Printer* printer) const = 0; @@ -144,7 +153,7 @@ class FieldGenerator { // Convenience class which constructs FieldGenerators for a Descriptor. class FieldGeneratorMap { public: - explicit FieldGeneratorMap(const Descriptor* descriptor); + explicit FieldGeneratorMap(const Descriptor* descriptor, const Options& options); ~FieldGeneratorMap(); const FieldGenerator& get(const FieldDescriptor* field) const; @@ -153,7 +162,8 @@ class FieldGeneratorMap { const Descriptor* descriptor_; scoped_array<scoped_ptr<FieldGenerator> > field_generators_; - static FieldGenerator* MakeGenerator(const FieldDescriptor* field); + static FieldGenerator* MakeGenerator(const FieldDescriptor* field, + const Options& options); GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FieldGeneratorMap); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.cc index 3e6d414..4eb89f4 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.cc @@ -51,7 +51,7 @@ namespace cpp { // =================================================================== FileGenerator::FileGenerator(const FileDescriptor* file, - const string& dllexport_decl) + const Options& options) : file_(file), message_generators_( new scoped_ptr<MessageGenerator>[file->message_type_count()]), @@ -61,26 +61,26 @@ FileGenerator::FileGenerator(const FileDescriptor* file, new scoped_ptr<ServiceGenerator>[file->service_count()]), extension_generators_( new scoped_ptr<ExtensionGenerator>[file->extension_count()]), - dllexport_decl_(dllexport_decl) { + options_(options) { for (int i = 0; i < file->message_type_count(); i++) { message_generators_[i].reset( - new MessageGenerator(file->message_type(i), dllexport_decl)); + new MessageGenerator(file->message_type(i), options)); } for (int i = 0; i < file->enum_type_count(); i++) { enum_generators_[i].reset( - new EnumGenerator(file->enum_type(i), dllexport_decl)); + new EnumGenerator(file->enum_type(i), options)); } for (int i = 0; i < file->service_count(); i++) { service_generators_[i].reset( - new ServiceGenerator(file->service(i), dllexport_decl)); + new ServiceGenerator(file->service(i), options)); } for (int i = 0; i < file->extension_count(); i++) { extension_generators_[i].reset( - new ExtensionGenerator(file->extension(i), dllexport_decl)); + new ExtensionGenerator(file->extension(i), options)); } SplitStringUsing(file_->package(), ".", &package_parts_); @@ -104,6 +104,7 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { "filename", file_->name(), "filename_identifier", filename_identifier); + printer->Print( "#include <google/protobuf/stubs/common.h>\n" "\n"); @@ -128,7 +129,17 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { // OK, it's now safe to #include other files. printer->Print( - "#include <google/protobuf/generated_message_util.h>\n" + "#include <google/protobuf/generated_message_util.h>\n"); + if (file_->message_type_count() > 0) { + if (HasDescriptorMethods(file_)) { + printer->Print( + "#include <google/protobuf/message.h>\n"); + } else { + printer->Print( + "#include <google/protobuf/message_lite.h>\n"); + } + } + printer->Print( "#include <google/protobuf/repeated_field.h>\n" "#include <google/protobuf/extension_set.h>\n"); @@ -137,9 +148,9 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { "#include <google/protobuf/unknown_field_set.h>\n"); } - if (HasDescriptorMethods(file_)) { + if (HasDescriptorMethods(file_) && HasEnumDefinitions(file_)) { printer->Print( - "#include <google/protobuf/generated_message_reflection.h>\n"); + "#include <google/protobuf/generated_enum_reflection.h>\n"); } if (HasGenericServices(file_)) { @@ -147,6 +158,11 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { "#include <google/protobuf/service.h>\n"); } + if (HasUnknownFields(file_) && file_->message_type_count() > 0) { + printer->Print( + "#include <google/protobuf/unknown_field_set.h>\n"); + } + for (int i = 0; i < file_->dependency_count(); i++) { printer->Print( @@ -154,9 +170,11 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { "dependency", StripProto(file_->dependency(i)->name())); } + printer->Print( "// @@protoc_insertion_point(includes)\n"); + // Open namespace. GenerateNamespaceOpeners(printer); @@ -167,7 +185,7 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { "// Internal implementation detail -- do not call these.\n" "void $dllexport_decl$ $adddescriptorsname$();\n", "adddescriptorsname", GlobalAddDescriptorsName(file_->name()), - "dllexport_decl", dllexport_decl_); + "dllexport_decl", options_.dllexport_decl); printer->Print( // Note that we don't put dllexport_decl on these because they are only @@ -287,6 +305,7 @@ void FileGenerator::GenerateHeader(io::Printer* printer) { void FileGenerator::GenerateSource(io::Printer* printer) { printer->Print( "// Generated by the protocol buffer compiler. DO NOT EDIT!\n" + "// source: $filename$\n" "\n" // The generated code calls accessors that might be deprecated. We don't @@ -296,14 +315,17 @@ void FileGenerator::GenerateSource(io::Printer* printer) { "\n" "#include <algorithm>\n" // for swap() "\n" + "#include <google/protobuf/stubs/common.h>\n" "#include <google/protobuf/stubs/once.h>\n" "#include <google/protobuf/io/coded_stream.h>\n" "#include <google/protobuf/wire_format_lite_inl.h>\n", + "filename", file_->name(), "basename", StripProto(file_->name())); if (HasDescriptorMethods(file_)) { printer->Print( "#include <google/protobuf/descriptor.h>\n" + "#include <google/protobuf/generated_message_reflection.h>\n" "#include <google/protobuf/reflection_ops.h>\n" "#include <google/protobuf/wire_format.h>\n"); } @@ -509,10 +531,12 @@ void FileGenerator::GenerateBuildDescriptors(io::Printer* printer) { " static bool already_here = false;\n" " if (already_here) return;\n" " already_here = true;\n" - " GOOGLE_PROTOBUF_VERIFY_VERSION;\n", + " GOOGLE_PROTOBUF_VERIFY_VERSION;\n" + "\n", // Without. "void $adddescriptorsname$_impl() {\n" - " GOOGLE_PROTOBUF_VERIFY_VERSION;\n", + " GOOGLE_PROTOBUF_VERIFY_VERSION;\n" + "\n", // Vars. "adddescriptorsname", GlobalAddDescriptorsName(file_->name())); @@ -526,9 +550,9 @@ void FileGenerator::GenerateBuildDescriptors(io::Printer* printer) { vector<string> dependency_package_parts; SplitStringUsing(dependency->package(), ".", &dependency_package_parts); printer->Print("::"); - for (int i = 0; i < dependency_package_parts.size(); i++) { + for (int j = 0; j < dependency_package_parts.size(); j++) { printer->Print("$name$::", - "name", dependency_package_parts[i]); + "name", dependency_package_parts[j]); } // Call its AddDescriptors function. printer->Print( @@ -552,10 +576,12 @@ void FileGenerator::GenerateBuildDescriptors(io::Printer* printer) { static const int kBytesPerLine = 40; for (int i = 0; i < file_data.size(); i += kBytesPerLine) { printer->Print("\n \"$data$\"", - "data", EscapeTrigraphs(CEscape(file_data.substr(i, kBytesPerLine)))); + "data", + EscapeTrigraphs( + CEscape(file_data.substr(i, kBytesPerLine)))); } printer->Print( - ", $size$);\n", + ", $size$);\n", "size", SimpleItoa(file_data.size())); // Call MessageFactory::InternalRegisterGeneratedFile(). diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.h index b4e0128..2deefaa 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_file.h @@ -39,6 +39,7 @@ #include <vector> #include <google/protobuf/stubs/common.h> #include <google/protobuf/compiler/cpp/cpp_field.h> +#include <google/protobuf/compiler/cpp/cpp_options.h> namespace google { namespace protobuf { @@ -61,7 +62,7 @@ class FileGenerator { public: // See generator.cc for the meaning of dllexport_decl. explicit FileGenerator(const FileDescriptor* file, - const string& dllexport_decl); + const Options& options); ~FileGenerator(); void GenerateHeader(io::Printer* printer); @@ -85,7 +86,7 @@ class FileGenerator { // E.g. if the package is foo.bar, package_parts_ is {"foo", "bar"}. vector<string> package_parts_; - string dllexport_decl_; + const Options options_; GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FileGenerator); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_generator.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_generator.cc index bb84e2a..1813510 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_generator.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_generator.cc @@ -78,11 +78,13 @@ bool CppGenerator::Generate(const FileDescriptor* file, // } // FOO_EXPORT is a macro which should expand to __declspec(dllexport) or // __declspec(dllimport) depending on what is being compiled. - string dllexport_decl; + Options file_options; for (int i = 0; i < options.size(); i++) { if (options[i].first == "dllexport_decl") { - dllexport_decl = options[i].second; + file_options.dllexport_decl = options[i].second; + } else if (options[i].first == "safe_boundary_check") { + file_options.safe_boundary_check = true; } else { *error = "Unknown generator option: " + options[i].first; return false; @@ -95,7 +97,7 @@ bool CppGenerator::Generate(const FileDescriptor* file, string basename = StripProto(file->name()); basename.append(".pb"); - FileGenerator file_generator(file, dllexport_decl); + FileGenerator file_generator(file, file_options); // Generate header. { diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.cc index c4b868c..c53d5d3 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.cc @@ -148,7 +148,7 @@ string ClassName(const Descriptor* descriptor, bool qualified) { string ClassName(const EnumDescriptor* enum_descriptor, bool qualified) { if (enum_descriptor->containing_type() == NULL) { if (qualified) { - return DotsToColons(enum_descriptor->full_name()); + return "::" + DotsToColons(enum_descriptor->full_name()); } else { return enum_descriptor->name(); } @@ -259,10 +259,27 @@ const char* DeclaredTypeMethodName(FieldDescriptor::Type type) { string DefaultValue(const FieldDescriptor* field) { switch (field->cpp_type()) { case FieldDescriptor::CPPTYPE_INT32: + // gcc rejects the decimal form of kint32min and kint64min. + if (field->default_value_int32() == kint32min) { + // Make sure we are in a 2's complement system. + GOOGLE_COMPILE_ASSERT( + (uint32)kint32min == (uint32)0 - (uint32)0x80000000, + kint32min_value_error); + return "-0x80000000"; + } return SimpleItoa(field->default_value_int32()); case FieldDescriptor::CPPTYPE_UINT32: return SimpleItoa(field->default_value_uint32()) + "u"; case FieldDescriptor::CPPTYPE_INT64: + // See the comments for CPPTYPE_INT32. + if (field->default_value_int64() == kint64min) { + // Make sure we are in a 2's complement system. + GOOGLE_COMPILE_ASSERT( + (uint64)kint64min == + (uint64)0 - (uint64)GOOGLE_LONGLONG(0x8000000000000000), + kint64min_value_error); + return "GOOGLE_LONGLONG(-0x8000000000000000)"; + } return "GOOGLE_LONGLONG(" + SimpleItoa(field->default_value_int64()) + ")"; case FieldDescriptor::CPPTYPE_UINT64: return "GOOGLE_ULONGLONG(" + SimpleItoa(field->default_value_uint64())+ ")"; @@ -308,8 +325,9 @@ string DefaultValue(const FieldDescriptor* field) { ClassName(field->enum_type(), true), field->default_value_enum()->number()); case FieldDescriptor::CPPTYPE_STRING: - return "\"" + EscapeTrigraphs(CEscape(field->default_value_string())) + - "\""; + return "\"" + EscapeTrigraphs( + CEscape(field->default_value_string())) + + "\""; case FieldDescriptor::CPPTYPE_MESSAGE: return FieldMessageTypeName(field) + "::default_instance()"; } @@ -401,6 +419,23 @@ void PrintHandlingOptionalStaticInitializers( } } + +static bool HasEnumDefinitions(const Descriptor* message_type) { + if (message_type->enum_type_count() > 0) return true; + for (int i = 0; i < message_type->nested_type_count(); ++i) { + if (HasEnumDefinitions(message_type->nested_type(i))) return true; + } + return false; +} + +bool HasEnumDefinitions(const FileDescriptor* file) { + if (file->enum_type_count() > 0) return true; + for (int i = 0; i < file->message_type_count(); ++i) { + if (HasEnumDefinitions(file->message_type(i))) return true; + } + return false; +} + } // namespace cpp } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.h index 690fbec..1b5b1ac 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_helpers.h @@ -122,24 +122,27 @@ string GlobalShutdownFileName(const string& filename); string EscapeTrigraphs(const string& to_escape); // Do message classes in this file keep track of unknown fields? -inline bool HasUnknownFields(const FileDescriptor *file) { +inline bool HasUnknownFields(const FileDescriptor* file) { return file->options().optimize_for() != FileOptions::LITE_RUNTIME || file->options().retain_unknown_fields(); } +// Does this file have any enum type definitions? +bool HasEnumDefinitions(const FileDescriptor* file); + // Does this file have generated parsing, serialization, and other // standard methods for which reflection-based fallback implementations exist? -inline bool HasGeneratedMethods(const FileDescriptor *file) { +inline bool HasGeneratedMethods(const FileDescriptor* file) { return file->options().optimize_for() != FileOptions::CODE_SIZE; } // Do message classes in this file have descriptor and refelction methods? -inline bool HasDescriptorMethods(const FileDescriptor *file) { +inline bool HasDescriptorMethods(const FileDescriptor* file) { return file->options().optimize_for() != FileOptions::LITE_RUNTIME; } // Should we generate generic services for this file? -inline bool HasGenericServices(const FileDescriptor *file) { +inline bool HasGenericServices(const FileDescriptor* file) { return file->service_count() > 0 && file->options().optimize_for() != FileOptions::LITE_RUNTIME && file->options().cc_generic_services(); @@ -174,6 +177,7 @@ void PrintHandlingOptionalStaticInitializers( io::Printer* printer, const char* with_static_init, const char* without_static_init); + } // namespace cpp } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc index 4248747..a7a81b0 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.cc @@ -48,6 +48,7 @@ #include <google/protobuf/wire_format.h> #include <google/protobuf/descriptor.pb.h> + namespace google { namespace protobuf { namespace compiler { @@ -103,6 +104,13 @@ struct ExtensionRangeSorter { } }; +// Returns true if the "required" restriction check should be ignored for the +// given field. +inline static bool ShouldIgnoreRequiredFieldCheck( + const FieldDescriptor* field) { + return false; +} + // Returns true if the message type has any required fields. If it doesn't, // we can optimize out calls to its IsInitialized() method. // @@ -129,7 +137,8 @@ static bool HasRequiredFields( if (field->is_required()) { return true; } - if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { + if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE && + !ShouldIgnoreRequiredFieldCheck(field)) { if (HasRequiredFields(field->message_type(), already_seen)) { return true; } @@ -280,11 +289,11 @@ void OptimizePadding(vector<const FieldDescriptor*>* fields) { // =================================================================== MessageGenerator::MessageGenerator(const Descriptor* descriptor, - const string& dllexport_decl) + const Options& options) : descriptor_(descriptor), classname_(ClassName(descriptor, false)), - dllexport_decl_(dllexport_decl), - field_generators_(descriptor), + options_(options), + field_generators_(descriptor, options), nested_generators_(new scoped_ptr<MessageGenerator>[ descriptor->nested_type_count()]), enum_generators_(new scoped_ptr<EnumGenerator>[ @@ -294,17 +303,17 @@ MessageGenerator::MessageGenerator(const Descriptor* descriptor, for (int i = 0; i < descriptor->nested_type_count(); i++) { nested_generators_[i].reset( - new MessageGenerator(descriptor->nested_type(i), dllexport_decl)); + new MessageGenerator(descriptor->nested_type(i), options)); } for (int i = 0; i < descriptor->enum_type_count(); i++) { enum_generators_[i].reset( - new EnumGenerator(descriptor->enum_type(i), dllexport_decl)); + new EnumGenerator(descriptor->enum_type(i), options)); } for (int i = 0; i < descriptor->extension_count(); i++) { extension_generators_[i].reset( - new ExtensionGenerator(descriptor->extension(i), dllexport_decl)); + new ExtensionGenerator(descriptor->extension(i), options)); } } @@ -349,7 +358,7 @@ GenerateFieldAccessorDeclarations(io::Printer* printer) { PrintFieldComment(printer, field); map<string, string> vars; - SetCommonFieldVariables(field, &vars); + SetCommonFieldVariables(field, &vars, options_); vars["constant_name"] = FieldConstantName(field); if (field->is_repeated()) { @@ -386,7 +395,7 @@ GenerateFieldAccessorDefinitions(io::Printer* printer) { PrintFieldComment(printer, field); map<string, string> vars; - SetCommonFieldVariables(field, &vars); + SetCommonFieldVariables(field, &vars, options_); // Generate has_$name$() or $name$_size(). if (field->is_repeated()) { @@ -446,10 +455,10 @@ GenerateClassDefinition(io::Printer* printer) { map<string, string> vars; vars["classname"] = classname_; vars["field_count"] = SimpleItoa(descriptor_->field_count()); - if (dllexport_decl_.empty()) { + if (options_.dllexport_decl.empty()) { vars["dllexport"] = ""; } else { - vars["dllexport"] = dllexport_decl_ + " "; + vars["dllexport"] = options_.dllexport_decl + " "; } vars["superclass"] = SuperClassName(descriptor_); @@ -507,6 +516,7 @@ GenerateClassDefinition(io::Printer* printer) { "\n"); } + printer->Print(vars, "void Swap($classname$* other);\n" "\n" @@ -605,6 +615,7 @@ GenerateClassDefinition(io::Printer* printer) { printer->Print(" private:\n"); printer->Indent(); + for (int i = 0; i < descriptor_->field_count(); i++) { if (!descriptor_->field(i)->is_repeated()) { printer->Print( @@ -680,7 +691,7 @@ GenerateClassDefinition(io::Printer* printer) { // Without. "friend void $dllexport_decl$ $adddescriptorsname$_impl();\n", // Vars. - "dllexport_decl", dllexport_decl_, + "dllexport_decl", options_.dllexport_decl, "adddescriptorsname", GlobalAddDescriptorsName(descriptor_->file()->name())); @@ -772,9 +783,11 @@ GenerateDescriptorInitializer(io::Printer* printer, int index) { printer->Print(vars, " -1,\n"); } + printer->Print( + " ::google::protobuf::DescriptorPool::generated_pool(),\n"); + printer->Print(vars, + " ::google::protobuf::MessageFactory::generated_factory(),\n"); printer->Print(vars, - " ::google::protobuf::DescriptorPool::generated_pool(),\n" - " ::google::protobuf::MessageFactory::generated_factory(),\n" " sizeof($classname$));\n"); // Handle nested types. @@ -803,6 +816,13 @@ GenerateTypeRegistrations(io::Printer* printer) { void MessageGenerator:: GenerateDefaultInstanceAllocator(io::Printer* printer) { + // Construct the default instances of all fields, as they will be used + // when creating the default instance of the entire message. + for (int i = 0; i < descriptor_->field_count(); i++) { + field_generators_.get(descriptor_->field(i)) + .GenerateDefaultInstanceAllocator(printer); + } + // Construct the default instance. We can't call InitAsDefaultInstance() yet // because we need to make sure all default instances that this one might // depend on are constructed first. @@ -846,6 +866,12 @@ GenerateShutdownCode(io::Printer* printer) { "classname", classname_); } + // Handle default instances of fields. + for (int i = 0; i < descriptor_->field_count(); i++) { + field_generators_.get(descriptor_->field(i)) + .GenerateShutdownCode(printer); + } + // Handle nested types. for (int i = 0; i < descriptor_->nested_type_count(); i++) { nested_generators_[i]->GenerateShutdownCode(printer); @@ -1956,6 +1982,7 @@ GenerateIsInitialized(io::Printer* printer) { for (int i = 0; i < descriptor_->field_count(); i++) { const FieldDescriptor* field = descriptor_->field(i); if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE && + !ShouldIgnoreRequiredFieldCheck(field) && HasRequiredFields(field->message_type())) { if (field->is_repeated()) { printer->Print( diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.h index 04778f6..a7e43d9 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message.h @@ -38,6 +38,7 @@ #include <string> #include <google/protobuf/stubs/common.h> #include <google/protobuf/compiler/cpp/cpp_field.h> +#include <google/protobuf/compiler/cpp/cpp_options.h> namespace google { namespace protobuf { @@ -57,7 +58,7 @@ class MessageGenerator { public: // See generator.cc for the meaning of dllexport_decl. explicit MessageGenerator(const Descriptor* descriptor, - const string& dllexport_decl); + const Options& options); ~MessageGenerator(); // Header stuff. @@ -153,7 +154,7 @@ class MessageGenerator { const Descriptor* descriptor_; string classname_; - string dllexport_decl_; + Options options_; FieldGeneratorMap field_generators_; scoped_array<scoped_ptr<MessageGenerator> > nested_generators_; scoped_array<scoped_ptr<EnumGenerator> > enum_generators_; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.cc index 7c785e7..447f975 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.cc @@ -45,8 +45,9 @@ namespace cpp { namespace { void SetMessageVariables(const FieldDescriptor* descriptor, - map<string, string>* variables) { - SetCommonFieldVariables(descriptor, variables); + map<string, string>* variables, + const Options& options) { + SetCommonFieldVariables(descriptor, variables, options); (*variables)["type"] = FieldMessageTypeName(descriptor); (*variables)["stream_writer"] = (*variables)["declared_type"] + (HasFastArraySerialization(descriptor->message_type()->file()) ? @@ -59,9 +60,10 @@ void SetMessageVariables(const FieldDescriptor* descriptor, // =================================================================== MessageFieldGenerator:: -MessageFieldGenerator(const FieldDescriptor* descriptor) +MessageFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetMessageVariables(descriptor, &variables_); + SetMessageVariables(descriptor, &variables_, options); } MessageFieldGenerator::~MessageFieldGenerator() {} @@ -76,7 +78,8 @@ GenerateAccessorDeclarations(io::Printer* printer) const { printer->Print(variables_, "inline const $type$& $name$() const$deprecation$;\n" "inline $type$* mutable_$name$()$deprecation$;\n" - "inline $type$* release_$name$()$deprecation$;\n"); + "inline $type$* release_$name$()$deprecation$;\n" + "inline void set_allocated_$name$($type$* $name$)$deprecation$;\n"); } void MessageFieldGenerator:: @@ -103,6 +106,15 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { " $type$* temp = $name$_;\n" " $name$_ = NULL;\n" " return temp;\n" + "}\n" + "inline void $classname$::set_allocated_$name$($type$* $name$) {\n" + " delete $name$_;\n" + " $name$_ = $name$;\n" + " if ($name$) {\n" + " set_has_$name$();\n" + " } else {\n" + " clear_has_$name$();\n" + " }\n" "}\n"); } @@ -167,9 +179,10 @@ GenerateByteSize(io::Printer* printer) const { // =================================================================== RepeatedMessageFieldGenerator:: -RepeatedMessageFieldGenerator(const FieldDescriptor* descriptor) +RepeatedMessageFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetMessageVariables(descriptor, &variables_); + SetMessageVariables(descriptor, &variables_, options); } RepeatedMessageFieldGenerator::~RepeatedMessageFieldGenerator() {} @@ -197,7 +210,7 @@ void RepeatedMessageFieldGenerator:: GenerateInlineAccessorDefinitions(io::Printer* printer) const { printer->Print(variables_, "inline const $type$& $classname$::$name$(int index) const {\n" - " return $name$_.Get(index);\n" + " return $name$_.$cppget$(index);\n" "}\n" "inline $type$* $classname$::mutable_$name$(int index) {\n" " return $name$_.Mutable(index);\n" diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.h index f514727..a5ed68a 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_message_field.h @@ -46,7 +46,8 @@ namespace cpp { class MessageFieldGenerator : public FieldGenerator { public: - explicit MessageFieldGenerator(const FieldDescriptor* descriptor); + explicit MessageFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~MessageFieldGenerator(); // implements FieldGenerator --------------------------------------- @@ -71,7 +72,8 @@ class MessageFieldGenerator : public FieldGenerator { class RepeatedMessageFieldGenerator : public FieldGenerator { public: - explicit RepeatedMessageFieldGenerator(const FieldDescriptor* descriptor); + explicit RepeatedMessageFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~RepeatedMessageFieldGenerator(); // implements FieldGenerator --------------------------------------- diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_options.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_options.h new file mode 100644 index 0000000..7877066 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_options.h @@ -0,0 +1,58 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: rennie@google.com (Jeffrey Rennie) + +#ifndef GOOGLE_PROTOBUF_COMPILER_CPP_OPTIONS_H__ +#define GOOGLE_PROTOBUF_COMPILER_CPP_OPTIONS_H__ + +#include <string> + +#include <google/protobuf/stubs/common.h> +namespace google { +namespace protobuf { +namespace compiler { +namespace cpp { + +// Generator options: +struct Options { + Options() : safe_boundary_check(false) { + } + string dllexport_decl; + bool safe_boundary_check; +}; + +} // namespace cpp +} // namespace compiler +} // namespace protobuf + + +} // namespace google +#endif // GOOGLE_PROTOBUF_COMPILER_CPP_OPTIONS_H__ diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc index 5e8df0f..1c35fef 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.cc @@ -80,8 +80,9 @@ int FixedSize(FieldDescriptor::Type type) { } void SetPrimitiveVariables(const FieldDescriptor* descriptor, - map<string, string>* variables) { - SetCommonFieldVariables(descriptor, variables); + map<string, string>* variables, + const Options& options) { + SetCommonFieldVariables(descriptor, variables, options); (*variables)["type"] = PrimitiveTypeName(descriptor->cpp_type()); (*variables)["default"] = DefaultValue(descriptor); (*variables)["tag"] = SimpleItoa(internal::WireFormat::MakeTag(descriptor)); @@ -99,9 +100,10 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, // =================================================================== PrimitiveFieldGenerator:: -PrimitiveFieldGenerator(const FieldDescriptor* descriptor) +PrimitiveFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetPrimitiveVariables(descriptor, &variables_); + SetPrimitiveVariables(descriptor, &variables_, options); } PrimitiveFieldGenerator::~PrimitiveFieldGenerator() {} @@ -190,9 +192,10 @@ GenerateByteSize(io::Printer* printer) const { // =================================================================== RepeatedPrimitiveFieldGenerator:: -RepeatedPrimitiveFieldGenerator(const FieldDescriptor* descriptor) +RepeatedPrimitiveFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetPrimitiveVariables(descriptor, &variables_); + SetPrimitiveVariables(descriptor, &variables_, options); if (descriptor->options().packed()) { variables_["packed_reader"] = "ReadPackedPrimitive"; @@ -366,7 +369,9 @@ GenerateByteSize(io::Printer* printer) const { " total_size += $tag_size$ +\n" " ::google::protobuf::internal::WireFormatLite::Int32Size(data_size);\n" "}\n" + "GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN();\n" "_$name$_cached_byte_size_ = data_size;\n" + "GOOGLE_SAFE_CONCURRENT_WRITES_END();\n" "total_size += data_size;\n"); } else { printer->Print(variables_, diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.h index 8fcd74a..48249c4 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_primitive_field.h @@ -46,7 +46,8 @@ namespace cpp { class PrimitiveFieldGenerator : public FieldGenerator { public: - explicit PrimitiveFieldGenerator(const FieldDescriptor* descriptor); + explicit PrimitiveFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~PrimitiveFieldGenerator(); // implements FieldGenerator --------------------------------------- @@ -71,7 +72,8 @@ class PrimitiveFieldGenerator : public FieldGenerator { class RepeatedPrimitiveFieldGenerator : public FieldGenerator { public: - explicit RepeatedPrimitiveFieldGenerator(const FieldDescriptor* descriptor); + explicit RepeatedPrimitiveFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~RepeatedPrimitiveFieldGenerator(); // implements FieldGenerator --------------------------------------- diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.cc index c282568..d20018e 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.cc @@ -43,14 +43,14 @@ namespace compiler { namespace cpp { ServiceGenerator::ServiceGenerator(const ServiceDescriptor* descriptor, - const string& dllexport_decl) + const Options& options) : descriptor_(descriptor) { vars_["classname"] = descriptor_->name(); vars_["full_name"] = descriptor_->full_name(); - if (dllexport_decl.empty()) { + if (options.dllexport_decl.empty()) { vars_["dllexport"] = ""; } else { - vars_["dllexport"] = dllexport_decl + " "; + vars_["dllexport"] = options.dllexport_decl + " "; } } diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.h index 10e9dd3..820f9f5 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_service.h @@ -38,6 +38,7 @@ #include <map> #include <string> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/compiler/cpp/cpp_options.h> #include <google/protobuf/descriptor.h> namespace google { @@ -55,7 +56,7 @@ class ServiceGenerator { public: // See generator.cc for the meaning of dllexport_decl. explicit ServiceGenerator(const ServiceDescriptor* descriptor, - const string& dllexport_decl); + const Options& options); ~ServiceGenerator(); // Header stuff. diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.cc index 3cec2b6..9c0911a 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.cc @@ -46,11 +46,14 @@ namespace cpp { namespace { void SetStringVariables(const FieldDescriptor* descriptor, - map<string, string>* variables) { - SetCommonFieldVariables(descriptor, variables); + map<string, string>* variables, + const Options& options) { + SetCommonFieldVariables(descriptor, variables, options); (*variables)["default"] = DefaultValue(descriptor); + (*variables)["default_length"] = + SimpleItoa(descriptor->default_value_string().length()); (*variables)["default_variable"] = descriptor->default_value_string().empty() - ? "::google::protobuf::internal::kEmptyString" + ? "&::google::protobuf::internal::kEmptyString" : "_default_" + FieldName(descriptor) + "_"; (*variables)["pointer_type"] = descriptor->type() == FieldDescriptor::TYPE_BYTES ? "void" : "char"; @@ -61,9 +64,10 @@ void SetStringVariables(const FieldDescriptor* descriptor, // =================================================================== StringFieldGenerator:: -StringFieldGenerator(const FieldDescriptor* descriptor) +StringFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetStringVariables(descriptor, &variables_); + SetStringVariables(descriptor, &variables_, options); } StringFieldGenerator::~StringFieldGenerator() {} @@ -72,7 +76,7 @@ void StringFieldGenerator:: GeneratePrivateMembers(io::Printer* printer) const { printer->Print(variables_, "::std::string* $name$_;\n"); if (!descriptor_->default_value_string().empty()) { - printer->Print(variables_, "static const ::std::string $default_variable$;\n"); + printer->Print(variables_, "static ::std::string* $default_variable$;\n"); } } @@ -109,7 +113,9 @@ GenerateAccessorDeclarations(io::Printer* printer) const { "inline void set_$name$(const $pointer_type$* value, size_t size)" "$deprecation$;\n" "inline ::std::string* mutable_$name$()$deprecation$;\n" - "inline ::std::string* release_$name$()$deprecation$;\n"); + "inline ::std::string* release_$name$()$deprecation$;\n" + "inline void set_allocated_$name$(::std::string* $name$)$deprecation$;\n"); + if (descriptor_->options().ctype() != FieldOptions::STRING) { printer->Outdent(); @@ -126,14 +132,14 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { "}\n" "inline void $classname$::set_$name$(const ::std::string& value) {\n" " set_has_$name$();\n" - " if ($name$_ == &$default_variable$) {\n" + " if ($name$_ == $default_variable$) {\n" " $name$_ = new ::std::string;\n" " }\n" " $name$_->assign(value);\n" "}\n" "inline void $classname$::set_$name$(const char* value) {\n" " set_has_$name$();\n" - " if ($name$_ == &$default_variable$) {\n" + " if ($name$_ == $default_variable$) {\n" " $name$_ = new ::std::string;\n" " }\n" " $name$_->assign(value);\n" @@ -141,20 +147,20 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { "inline " "void $classname$::set_$name$(const $pointer_type$* value, size_t size) {\n" " set_has_$name$();\n" - " if ($name$_ == &$default_variable$) {\n" + " if ($name$_ == $default_variable$) {\n" " $name$_ = new ::std::string;\n" " }\n" " $name$_->assign(reinterpret_cast<const char*>(value), size);\n" "}\n" "inline ::std::string* $classname$::mutable_$name$() {\n" " set_has_$name$();\n" - " if ($name$_ == &$default_variable$) {\n"); + " if ($name$_ == $default_variable$) {\n"); if (descriptor_->default_value_string().empty()) { printer->Print(variables_, " $name$_ = new ::std::string;\n"); } else { printer->Print(variables_, - " $name$_ = new ::std::string($default_variable$);\n"); + " $name$_ = new ::std::string(*$default_variable$);\n"); } printer->Print(variables_, " }\n" @@ -162,21 +168,34 @@ GenerateInlineAccessorDefinitions(io::Printer* printer) const { "}\n" "inline ::std::string* $classname$::release_$name$() {\n" " clear_has_$name$();\n" - " if ($name$_ == &$default_variable$) {\n" + " if ($name$_ == $default_variable$) {\n" " return NULL;\n" " } else {\n" " ::std::string* temp = $name$_;\n" - " $name$_ = const_cast< ::std::string*>(&$default_variable$);\n" + " $name$_ = const_cast< ::std::string*>($default_variable$);\n" " return temp;\n" " }\n" + "}\n" + "inline void $classname$::set_allocated_$name$(::std::string* $name$) {\n" + " if ($name$_ != $default_variable$) {\n" + " delete $name$_;\n" + " }\n" + " if ($name$) {\n" + " set_has_$name$();\n" + " $name$_ = $name$;\n" + " } else {\n" + " clear_has_$name$();\n" + " $name$_ = const_cast< ::std::string*>($default_variable$);\n" + " }\n" "}\n"); } void StringFieldGenerator:: GenerateNonInlineAccessorDefinitions(io::Printer* printer) const { if (!descriptor_->default_value_string().empty()) { + // Initialized in GenerateDefaultInstanceAllocator. printer->Print(variables_, - "const ::std::string $classname$::$default_variable$($default$);\n"); + "::std::string* $classname$::$default_variable$ = NULL;\n"); } } @@ -184,13 +203,13 @@ void StringFieldGenerator:: GenerateClearingCode(io::Printer* printer) const { if (descriptor_->default_value_string().empty()) { printer->Print(variables_, - "if ($name$_ != &$default_variable$) {\n" + "if ($name$_ != $default_variable$) {\n" " $name$_->clear();\n" "}\n"); } else { printer->Print(variables_, - "if ($name$_ != &$default_variable$) {\n" - " $name$_->assign($default_variable$);\n" + "if ($name$_ != $default_variable$) {\n" + " $name$_->assign(*$default_variable$);\n" "}\n"); } } @@ -208,18 +227,35 @@ GenerateSwappingCode(io::Printer* printer) const { void StringFieldGenerator:: GenerateConstructorCode(io::Printer* printer) const { printer->Print(variables_, - "$name$_ = const_cast< ::std::string*>(&$default_variable$);\n"); + "$name$_ = const_cast< ::std::string*>($default_variable$);\n"); } void StringFieldGenerator:: GenerateDestructorCode(io::Printer* printer) const { printer->Print(variables_, - "if ($name$_ != &$default_variable$) {\n" + "if ($name$_ != $default_variable$) {\n" " delete $name$_;\n" "}\n"); } void StringFieldGenerator:: +GenerateDefaultInstanceAllocator(io::Printer* printer) const { + if (!descriptor_->default_value_string().empty()) { + printer->Print(variables_, + "$classname$::$default_variable$ =\n" + " new ::std::string($default$, $default_length$);\n"); + } +} + +void StringFieldGenerator:: +GenerateShutdownCode(io::Printer* printer) const { + if (!descriptor_->default_value_string().empty()) { + printer->Print(variables_, + "delete $classname$::$default_variable$;\n"); + } +} + +void StringFieldGenerator:: GenerateMergeFromCodedStream(io::Printer* printer) const { printer->Print(variables_, "DO_(::google::protobuf::internal::WireFormatLite::Read$declared_type$(\n" @@ -273,9 +309,10 @@ GenerateByteSize(io::Printer* printer) const { // =================================================================== RepeatedStringFieldGenerator:: -RepeatedStringFieldGenerator(const FieldDescriptor* descriptor) +RepeatedStringFieldGenerator(const FieldDescriptor* descriptor, + const Options& options) : descriptor_(descriptor) { - SetStringVariables(descriptor, &variables_); + SetStringVariables(descriptor, &variables_, options); } RepeatedStringFieldGenerator::~RepeatedStringFieldGenerator() {} @@ -328,7 +365,7 @@ void RepeatedStringFieldGenerator:: GenerateInlineAccessorDefinitions(io::Printer* printer) const { printer->Print(variables_, "inline const ::std::string& $classname$::$name$(int index) const {\n" - " return $name$_.Get(index);\n" + " return $name$_.$cppget$(index);\n" "}\n" "inline ::std::string* $classname$::mutable_$name$(int index) {\n" " return $name$_.Mutable(index);\n" diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.h index 7f45107..3264134 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_string_field.h @@ -46,7 +46,8 @@ namespace cpp { class StringFieldGenerator : public FieldGenerator { public: - explicit StringFieldGenerator(const FieldDescriptor* descriptor); + explicit StringFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~StringFieldGenerator(); // implements FieldGenerator --------------------------------------- @@ -59,6 +60,8 @@ class StringFieldGenerator : public FieldGenerator { void GenerateSwappingCode(io::Printer* printer) const; void GenerateConstructorCode(io::Printer* printer) const; void GenerateDestructorCode(io::Printer* printer) const; + void GenerateDefaultInstanceAllocator(io::Printer* printer) const; + void GenerateShutdownCode(io::Printer* printer) const; void GenerateMergeFromCodedStream(io::Printer* printer) const; void GenerateSerializeWithCachedSizes(io::Printer* printer) const; void GenerateSerializeWithCachedSizesToArray(io::Printer* printer) const; @@ -73,7 +76,8 @@ class StringFieldGenerator : public FieldGenerator { class RepeatedStringFieldGenerator : public FieldGenerator { public: - explicit RepeatedStringFieldGenerator(const FieldDescriptor* descriptor); + explicit RepeatedStringFieldGenerator(const FieldDescriptor* descriptor, + const Options& options); ~RepeatedStringFieldGenerator(); // implements FieldGenerator --------------------------------------- diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto index 54d830f..e14a818 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_test_bad_identifiers.proto @@ -103,9 +103,19 @@ message TestConflictingSymbolNames { message DO {} optional DO do = 32; + // Some template parameter names for extensions. + optional int32 field_type = 33; + optional bool is_packed = 34; + extensions 1000 to max; } +message TestConflictingSymbolNamesExtension { + extend TestConflictingSymbolNames { + repeated int32 repeated_int32_ext = 20423638 [packed=true]; + } +} + message DummyMessage {} service TestConflictingMethodNames { diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.cc b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.cc index 301a7ce..b5f9ab58 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.cc @@ -44,6 +44,8 @@ // correctly and produces the interfaces we expect, which is why this test // is written this way. +#include <google/protobuf/compiler/cpp/cpp_unittest.h> + #include <vector> #include <google/protobuf/unittest.pb.h> @@ -64,7 +66,7 @@ #include <google/protobuf/stubs/substitute.h> #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -74,6 +76,8 @@ namespace cpp { // Can't use an anonymous namespace here due to brokenness of Tru64 compiler. namespace cpp_unittest { +namespace protobuf_unittest = ::protobuf_unittest; + class MockErrorCollector : public MultiFileErrorCollector { public: @@ -174,6 +178,15 @@ TEST(GeneratedMessageTest, Trigraph) { EXPECT_EQ("? ? ?? ?? ??? ?\?/ ?\?-", extreme_default.cpp_trigraph()); } +TEST(GeneratedMessageTest, ExtremeSmallIntegerDefault) { + const unittest::TestExtremeDefaultValues& extreme_default = + unittest::TestExtremeDefaultValues::default_instance(); + EXPECT_EQ(-0x80000000, kint32min); + EXPECT_EQ(GOOGLE_LONGLONG(-0x8000000000000000), kint64min); + EXPECT_EQ(kint32min, extreme_default.really_small_int32()); + EXPECT_EQ(kint64min, extreme_default.really_small_int64()); +} + TEST(GeneratedMessageTest, Accessors) { // Set every field to a unique value then go back and check all those // values. @@ -202,6 +215,13 @@ TEST(GeneratedMessageTest, MutableStringDefault) { EXPECT_EQ("hello", *message.mutable_default_string()); } +TEST(GeneratedMessageTest, StringDefaults) { + unittest::TestExtremeDefaultValues message; + // Check if '\000' can be used in default string value. + EXPECT_EQ(string("hel\000lo", 6), message.string_with_zero()); + EXPECT_EQ(string("wor\000ld", 6), message.bytes_with_zero()); +} + TEST(GeneratedMessageTest, ReleaseString) { // Check that release_foo() starts out NULL, and gives us a value // that we can delete after it's been set. @@ -244,6 +264,49 @@ TEST(GeneratedMessageTest, ReleaseMessage) { EXPECT_FALSE(message.has_optional_nested_message()); } +TEST(GeneratedMessageTest, SetAllocatedString) { + // Check that set_allocated_foo() works for strings. + unittest::TestAllTypes message; + + EXPECT_FALSE(message.has_optional_string()); + const string kHello("hello"); + message.set_optional_string(kHello); + EXPECT_TRUE(message.has_optional_string()); + + message.set_allocated_optional_string(NULL); + EXPECT_FALSE(message.has_optional_string()); + EXPECT_EQ("", message.optional_string()); + + message.set_allocated_optional_string(new string(kHello)); + EXPECT_TRUE(message.has_optional_string()); + EXPECT_EQ(kHello, message.optional_string()); +} + +TEST(GeneratedMessageTest, SetAllocatedMessage) { + // Check that set_allocated_foo() can be called in all cases. + unittest::TestAllTypes message; + + EXPECT_FALSE(message.has_optional_nested_message()); + + message.mutable_optional_nested_message()->set_bb(1); + EXPECT_TRUE(message.has_optional_nested_message()); + + message.set_allocated_optional_nested_message(NULL); + EXPECT_FALSE(message.has_optional_nested_message()); + EXPECT_EQ(&unittest::TestAllTypes::NestedMessage::default_instance(), + &message.optional_nested_message()); + + message.mutable_optional_nested_message()->set_bb(1); + unittest::TestAllTypes::NestedMessage* nest = + message.release_optional_nested_message(); + ASSERT_TRUE(nest != NULL); + EXPECT_FALSE(message.has_optional_nested_message()); + + message.set_allocated_optional_nested_message(nest); + EXPECT_TRUE(message.has_optional_nested_message()); + EXPECT_EQ(1, message.optional_nested_message().bb()); +} + TEST(GeneratedMessageTest, Clear) { // Set every field to a unique value, clear the message, then check that // it is cleared. @@ -695,6 +758,13 @@ TEST(GeneratedMessageTest, TestConflictingSymbolNames) { message.set_friend_(5); EXPECT_EQ(5, message.friend_()); + + // Instantiate extension template functions to test conflicting template + // parameter names. + typedef protobuf_unittest::TestConflictingSymbolNamesExtension ExtensionMessage; + message.AddExtension(ExtensionMessage::repeated_int32_ext, 123); + EXPECT_EQ(123, + message.GetExtension(ExtensionMessage::repeated_int32_ext, 0)); } #ifndef PROTOBUF_TEST_NO_DESCRIPTORS @@ -869,11 +939,10 @@ TEST(GeneratedEnumTest, MinAndMax) { EXPECT_NE(null_pointer, &unittest::ForeignEnum_MAX); EXPECT_NE(null_pointer, &unittest::ForeignEnum_ARRAYSIZE); - // Make sure we can use _MIN, _MAX and _ARRAYSIZE as switch cases. + // Make sure we can use _MIN and _MAX as switch cases. switch (unittest::SPARSE_A) { case unittest::TestSparseEnum_MIN: case unittest::TestSparseEnum_MAX: - case unittest::TestSparseEnum_ARRAYSIZE: break; default: break; diff --git a/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.h b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.h new file mode 100644 index 0000000..a3a1d1b --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/compiler/cpp/cpp_unittest.h @@ -0,0 +1,51 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// This header declares the namespace google::protobuf::protobuf_unittest in order to expose +// any problems with the generated class names. We use this header to ensure +// unittest.cc will declare the namespace prior to other includes, while obeying +// normal include ordering. +// +// When generating a class name of "foo.Bar" we must ensure we prefix the class +// name with "::", in case the namespace google::protobuf::foo exists. We intentionally +// trigger that case here by declaring google::protobuf::protobuf_unittest. +// +// See ClassName in helpers.h for more details. + +#ifndef GOOGLE_PROTOBUF_COMPILER_CPP_UNITTEST_H__ +#define GOOGLE_PROTOBUF_COMPILER_CPP_UNITTEST_H__ + +namespace google { +namespace protobuf { +namespace protobuf_unittest {} +} // namespace protobuf + +} // namespace google +#endif // GOOGLE_PROTOBUF_COMPILER_CPP_UNITTEST_H__ diff --git a/third_party/protobuf/src/google/protobuf/compiler/importer.h b/third_party/protobuf/src/google/protobuf/compiler/importer.h index 7a2efc2..7a62fa0 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/importer.h +++ b/third_party/protobuf/src/google/protobuf/compiler/importer.h @@ -280,8 +280,9 @@ class LIBPROTOBUF_EXPORT DiskSourceTree : public SourceTree { string virtual_path; string disk_path; - inline Mapping(const string& virtual_path, const string& disk_path) - : virtual_path(virtual_path), disk_path(disk_path) {} + inline Mapping(const string& virtual_path_param, + const string& disk_path_param) + : virtual_path(virtual_path_param), disk_path(disk_path_param) {} }; vector<Mapping> mappings_; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.cc new file mode 100644 index 0000000..60b4f2a --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.cc @@ -0,0 +1,236 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: kenton@google.com (Kenton Varda) +// Based on original Protocol Buffers design by +// Sanjay Ghemawat, Jeff Dean, and others. + +#include <google/protobuf/compiler/java/java_doc_comment.h> + +#include <vector> + +#include <google/protobuf/io/printer.h> +#include <google/protobuf/stubs/strutil.h> + +namespace google { +namespace protobuf { +namespace compiler { +namespace java { + +string EscapeJavadoc(const string& input) { + string result; + result.reserve(input.size() * 2); + + char prev = '*'; + + for (string::size_type i = 0; i < input.size(); i++) { + char c = input[i]; + switch (c) { + case '*': + // Avoid "/*". + if (prev == '/') { + result.append("*"); + } else { + result.push_back(c); + } + break; + case '/': + // Avoid "*/". + if (prev == '*') { + result.append("/"); + } else { + result.push_back(c); + } + break; + case '@': + // "{@" starts Javadoc markup. + if (prev == '{') { + result.append("@"); + } else { + result.push_back(c); + } + break; + case '<': + // Avoid interpretation as HTML. + result.append("<"); + break; + case '>': + // Avoid interpretation as HTML. + result.append(">"); + break; + case '&': + // Avoid interpretation as HTML. + result.append("&"); + break; + case '\\': + // Java interprets Unicode escape sequences anywhere! + result.append("\"); + break; + default: + result.push_back(c); + break; + } + + prev = c; + } + + return result; +} + +static void WriteDocCommentBodyForLocation( + io::Printer* printer, const SourceLocation& location) { + string comments = location.leading_comments.empty() ? + location.trailing_comments : location.leading_comments; + if (!comments.empty()) { + // TODO(kenton): Ideally we should parse the comment text as Markdown and + // write it back as HTML, but this requires a Markdown parser. For now + // we just use <pre> to get fixed-width text formatting. + + // If the comment itself contains block comment start or end markers, + // HTML-escape them so that they don't accidentally close the doc comment. + comments = EscapeJavadoc(comments); + + vector<string> lines; + SplitStringAllowEmpty(comments, "\n", &lines); + while (!lines.empty() && lines.back().empty()) { + lines.pop_back(); + } + + printer->Print( + " *\n" + " * <pre>\n"); + for (int i = 0; i < lines.size(); i++) { + // Most lines should start with a space. Watch out for lines that start + // with a /, since putting that right after the leading asterisk will + // close the comment. + if (!lines[i].empty() && lines[i][0] == '/') { + printer->Print(" * $line$\n", "line", lines[i]); + } else { + printer->Print(" *$line$\n", "line", lines[i]); + } + } + printer->Print(" * </pre>\n"); + } +} + +template <typename DescriptorType> +static void WriteDocCommentBody( + io::Printer* printer, const DescriptorType* descriptor) { + SourceLocation location; + if (descriptor->GetSourceLocation(&location)) { + WriteDocCommentBodyForLocation(printer, location); + } +} + +static string FirstLineOf(const string& value) { + string result = value; + + string::size_type pos = result.find_first_of('\n'); + if (pos != string::npos) { + result.erase(pos); + } + + // If line ends in an opening brace, make it "{ ... }" so it looks nice. + if (!result.empty() && result[result.size() - 1] == '{') { + result.append(" ... }"); + } + + return result; +} + +void WriteMessageDocComment(io::Printer* printer, const Descriptor* message) { + printer->Print( + "/**\n" + " * Protobuf type {@code $fullname$}\n", + "fullname", EscapeJavadoc(message->full_name())); + WriteDocCommentBody(printer, message); + printer->Print(" */\n"); +} + +void WriteFieldDocComment(io::Printer* printer, const FieldDescriptor* field) { + // In theory we should have slightly different comments for setters, getters, + // etc., but in practice everyone already knows the difference between these + // so it's redundant information. + + // We use the field declaration as the first line of the comment, e.g.: + // optional string foo = 5; + // This communicates a lot of information about the field in a small space. + // If the field is a group, the debug string might end with {. + printer->Print( + "/**\n" + " * <code>$def$</code>\n", + "def", EscapeJavadoc(FirstLineOf(field->DebugString()))); + WriteDocCommentBody(printer, field); + printer->Print(" */\n"); +} + +void WriteEnumDocComment(io::Printer* printer, const EnumDescriptor* enum_) { + printer->Print( + "/**\n" + " * Protobuf enum {@code $fullname$}\n", + "fullname", EscapeJavadoc(enum_->full_name())); + WriteDocCommentBody(printer, enum_); + printer->Print(" */\n"); +} + +void WriteEnumValueDocComment(io::Printer* printer, + const EnumValueDescriptor* value) { + printer->Print( + "/**\n" + " * <code>$def$</code>\n", + "def", EscapeJavadoc(FirstLineOf(value->DebugString()))); + WriteDocCommentBody(printer, value); + printer->Print(" */\n"); +} + +void WriteServiceDocComment(io::Printer* printer, + const ServiceDescriptor* service) { + printer->Print( + "/**\n" + " * Protobuf service {@code $fullname$}\n", + "fullname", EscapeJavadoc(service->full_name())); + WriteDocCommentBody(printer, service); + printer->Print(" */\n"); +} + +void WriteMethodDocComment(io::Printer* printer, + const MethodDescriptor* method) { + printer->Print( + "/**\n" + " * <code>$def$</code>\n", + "def", EscapeJavadoc(FirstLineOf(method->DebugString()))); + WriteDocCommentBody(printer, method); + printer->Print(" */\n"); +} + +} // namespace java +} // namespace compiler +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.h new file mode 100644 index 0000000..f77720b --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment.h @@ -0,0 +1,69 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: kenton@google.com (Kenton Varda) +// Based on original Protocol Buffers design by +// Sanjay Ghemawat, Jeff Dean, and others. + +#ifndef GOOGLE_PROTOBUF_COMPILER_JAVA_DOC_COMMENT_H__ +#define GOOGLE_PROTOBUF_COMPILER_JAVA_DOC_COMMENT_H__ + +#include <google/protobuf/descriptor.h> + +namespace google { +namespace protobuf { + namespace io { + class Printer; // printer.h + } +} + +namespace protobuf { +namespace compiler { +namespace java { + +void WriteMessageDocComment(io::Printer* printer, const Descriptor* message); +void WriteFieldDocComment(io::Printer* printer, const FieldDescriptor* field); +void WriteEnumDocComment(io::Printer* printer, const EnumDescriptor* enum_); +void WriteEnumValueDocComment(io::Printer* printer, + const EnumValueDescriptor* value); +void WriteServiceDocComment(io::Printer* printer, + const ServiceDescriptor* service); +void WriteMethodDocComment(io::Printer* printer, + const MethodDescriptor* method); + +// Exposed for testing only. +string EscapeJavadoc(const string& input); + +} // namespace java +} // namespace compiler +} // namespace protobuf + +} // namespace google +#endif // GOOGLE_PROTOBUF_COMPILER_JAVA_DOC_COMMENT_H__ diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment_unittest.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment_unittest.cc new file mode 100644 index 0000000..28b6d8b --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_doc_comment_unittest.cc @@ -0,0 +1,66 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: kenton@google.com (Kenton Varda) + +#include <google/protobuf/compiler/java/java_doc_comment.h> + +#include <gtest/gtest.h> + +namespace google { +namespace protobuf { +namespace compiler { +namespace java { +namespace { + +TEST(JavaDocCommentTest, Escaping) { + EXPECT_EQ("foo /* bar */ baz", EscapeJavadoc("foo /* bar */ baz")); + EXPECT_EQ("foo /*/ baz", EscapeJavadoc("foo /*/ baz")); + EXPECT_EQ("{@foo}", EscapeJavadoc("{@foo}")); + EXPECT_EQ("<i>&</i>", EscapeJavadoc("<i>&</i>")); + EXPECT_EQ("foo\u1234bar", EscapeJavadoc("foo\\u1234bar")); +} + +// TODO(kenton): It's hard to write a robust test of the doc comments -- we +// can only really compare the output against a golden value, which is a +// fairly tedious and fragile testing strategy. If we want to go that route, +// it probably makes sense to bite the bullet and write a test that compares +// the whole generated output for unittest.proto against a golden value, with +// a very simple script that can be run to regenerate it with the latest code. +// This would mean that updates to the golden file would have to be included +// in any change to the code generator, which would actually be fairly useful +// as it allows the reviewer to see clearly how the generated code is +// changing. + +} // namespace +} // namespace java +} // namespace compiler +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.cc index 9d7bcab..cfed815 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.cc @@ -36,6 +36,7 @@ #include <string> #include <google/protobuf/compiler/java/java_enum.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/descriptor.pb.h> @@ -67,6 +68,7 @@ EnumGenerator::EnumGenerator(const EnumDescriptor* descriptor) EnumGenerator::~EnumGenerator() {} void EnumGenerator::Generate(io::Printer* printer) { + WriteEnumDocComment(printer, descriptor_); if (HasDescriptorMethods(descriptor_)) { printer->Print( "public enum $classname$\n" @@ -85,6 +87,7 @@ void EnumGenerator::Generate(io::Printer* printer) { vars["name"] = canonical_values_[i]->name(); vars["index"] = SimpleItoa(canonical_values_[i]->index()); vars["number"] = SimpleItoa(canonical_values_[i]->number()); + WriteEnumValueDocComment(printer, canonical_values_[i]); printer->Print(vars, "$name$($index$, $number$),\n"); } @@ -100,6 +103,7 @@ void EnumGenerator::Generate(io::Printer* printer) { vars["classname"] = descriptor_->name(); vars["name"] = aliases_[i].value->name(); vars["canonical_name"] = aliases_[i].canonical_value->name(); + WriteEnumValueDocComment(printer, aliases_[i].value); printer->Print(vars, "public static final $classname$ $name$ = $canonical_name$;\n"); } @@ -108,6 +112,7 @@ void EnumGenerator::Generate(io::Printer* printer) { map<string, string> vars; vars["name"] = descriptor_->value(i)->name(); vars["number"] = SimpleItoa(descriptor_->value(i)->number()); + WriteEnumValueDocComment(printer, descriptor_->value(i)); printer->Print(vars, "public static final int $name$_VALUE = $number$;\n"); } @@ -187,19 +192,30 @@ void EnumGenerator::Generate(io::Printer* printer) { printer->Print( "}\n" "\n" - "private static final $classname$[] VALUES = {\n" - " ", + "private static final $classname$[] VALUES = ", "classname", descriptor_->name()); - for (int i = 0; i < descriptor_->value_count(); i++) { - printer->Print("$name$, ", - "name", descriptor_->value(i)->name()); + if (CanUseEnumValues()) { + // If the constants we are going to output are exactly the ones we + // have declared in the Java enum in the same order, then we can use + // the values() method that the Java compiler automatically generates + // for every enum. + printer->Print("values();\n"); + } else { + printer->Print( + "{\n" + " "); + for (int i = 0; i < descriptor_->value_count(); i++) { + printer->Print("$name$, ", + "name", descriptor_->value(i)->name()); + } + printer->Print( + "\n" + "};\n"); } printer->Print( "\n" - "};\n" - "\n" "public static $classname$ valueOf(\n" " com.google.protobuf.Descriptors.EnumValueDescriptor desc) {\n" " if (desc.getType() != getDescriptor()) {\n" @@ -237,6 +253,18 @@ void EnumGenerator::Generate(io::Printer* printer) { printer->Print("}\n\n"); } +bool EnumGenerator::CanUseEnumValues() { + if (canonical_values_.size() != descriptor_->value_count()) { + return false; + } + for (int i = 0; i < descriptor_->value_count(); i++) { + if (descriptor_->value(i)->name() != canonical_values_[i]->name()) { + return false; + } + } + return true; +} + } // namespace java } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.h index 05ece1f..9a9e574 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum.h @@ -73,6 +73,8 @@ class EnumGenerator { }; vector<Alias> aliases_; + bool CanUseEnumValues(); + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(EnumGenerator); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.cc index 72caa10..ec0b067 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.cc @@ -36,6 +36,7 @@ #include <string> #include <google/protobuf/compiler/java/java_enum_field.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/stubs/common.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> @@ -75,6 +76,7 @@ void SetEnumVariables(const FieldDescriptor* descriptor, // For singular messages and builders, one bit is used for the hasField bit. (*variables)["get_has_field_bit_message"] = GenerateGetBit(messageBitIndex); + (*variables)["set_has_field_bit_message"] = GenerateSetBit(messageBitIndex); (*variables)["get_has_field_bit_builder"] = GenerateGetBit(builderBitIndex); (*variables)["set_has_field_bit_builder"] = GenerateSetBit(builderBitIndex); @@ -86,6 +88,13 @@ void SetEnumVariables(const FieldDescriptor* descriptor, (*variables)["set_mutable_bit_builder"] = GenerateSetBit(builderBitIndex); (*variables)["clear_mutable_bit_builder"] = GenerateClearBit(builderBitIndex); + // For repeated fields, one bit is used for whether the array is immutable + // in the parsing constructor. + (*variables)["get_mutable_bit_parser"] = + GenerateGetBitMutableLocal(builderBitIndex); + (*variables)["set_mutable_bit_parser"] = + GenerateSetBitMutableLocal(builderBitIndex); + (*variables)["get_has_field_bit_from_local"] = GenerateGetBitFromLocal(builderBitIndex); (*variables)["set_has_field_bit_to_local"] = @@ -117,18 +126,25 @@ int EnumFieldGenerator::GetNumBitsForBuilder() const { void EnumFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$boolean has$capitalized_name$();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$boolean has$capitalized_name$();\n" "$deprecation$$type$ get$capitalized_name$();\n"); } void EnumFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private $type$ $name$_;\n" + "private $type$ $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_message$;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$() {\n" " return $name$_;\n" "}\n"); @@ -137,13 +153,19 @@ GenerateMembers(io::Printer* printer) const { void EnumFieldGenerator:: GenerateBuilderMembers(io::Printer* printer) const { printer->Print(variables_, - "private $type$ $name$_ = $default$;\n" + "private $type$ $name$_ = $default$;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_builder$;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$() {\n" " return $name$_;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$($type$ value) {\n" " if (value == null) {\n" " throw new NullPointerException();\n" @@ -152,7 +174,9 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_ = value;\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $clear_has_field_bit_builder$;\n" " $name$_ = $default$;\n" @@ -210,12 +234,17 @@ GenerateParsingCode(io::Printer* printer) const { "if (value != null) {\n"); } printer->Print(variables_, - " $set_has_field_bit_builder$;\n" + " $set_has_field_bit_message$;\n" " $name$_ = value;\n" "}\n"); } void EnumFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + // noop for enums +} + +void EnumFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { printer->Print(variables_, "if ($get_has_field_bit_message$) {\n" @@ -273,22 +302,33 @@ int RepeatedEnumFieldGenerator::GetNumBitsForBuilder() const { void RepeatedEnumFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$java.util.List<$type$> get$capitalized_name$List();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$int get$capitalized_name$Count();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$java.util.List<$type$> get$capitalized_name$List();\n" - "$deprecation$int get$capitalized_name$Count();\n" "$deprecation$$type$ get$capitalized_name$(int index);\n"); } void RepeatedEnumFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private java.util.List<$type$> $name$_;\n" + "private java.util.List<$type$> $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<$type$> get$capitalized_name$List() {\n" " return $name$_;\n" // note: unmodifiable list - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" "}\n"); @@ -320,21 +360,29 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_ = new java.util.ArrayList<$type$>($name$_);\n" " $set_mutable_bit_builder$;\n" " }\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, // Note: We return an unmodifiable list because otherwise the caller // could hold on to the returned list and modify it after the message // has been built, thus mutating the message which is supposed to be // immutable. "$deprecation$public java.util.List<$type$> get$capitalized_name$List() {\n" " return java.util.Collections.unmodifiableList($name$_);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$(\n" " int index, $type$ value) {\n" " if (value == null) {\n" @@ -344,7 +392,9 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_.set(index, value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder add$capitalized_name$($type$ value) {\n" " if (value == null) {\n" " throw new NullPointerException();\n" @@ -353,14 +403,18 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_.add(value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder addAll$capitalized_name$(\n" " java.lang.Iterable<? extends $type$> values) {\n" " ensure$capitalized_name$IsMutable();\n" " super.addAll(values, $name$_);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $name$_ = java.util.Collections.emptyList();\n" " $clear_mutable_bit_builder$;\n" @@ -434,7 +488,11 @@ GenerateParsingCode(io::Printer* printer) const { "if (value != null) {\n"); } printer->Print(variables_, - " add$capitalized_name$(value);\n" + " if (!$get_mutable_bit_parser$) {\n" + " $name$_ = new java.util.ArrayList<$type$>();\n" + " $set_mutable_bit_parser$;\n" + " }\n" + " $name$_.add(value);\n" "}\n"); } @@ -457,6 +515,14 @@ GenerateParsingCodeFromPacked(io::Printer* printer) const { } void RepeatedEnumFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + printer->Print(variables_, + "if ($get_mutable_bit_parser$) {\n" + " $name$_ = java.util.Collections.unmodifiableList($name$_);\n" + "}\n"); +} + +void RepeatedEnumFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { if (descriptor_->options().packed()) { printer->Print(variables_, diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.h index 0cad6be..90fae63 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_enum_field.h @@ -61,6 +61,7 @@ class EnumFieldGenerator : public FieldGenerator { void GenerateMergingCode(io::Printer* printer) const; void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; @@ -96,6 +97,7 @@ class RepeatedEnumFieldGenerator : public FieldGenerator { void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; void GenerateParsingCodeFromPacked(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_extension.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_extension.cc index 9b147c7..921fe65 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_extension.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_extension.cc @@ -33,6 +33,7 @@ // Sanjay Ghemawat, Jeff Dean, and others. #include <google/protobuf/compiler/java/java_extension.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/io/printer.h> @@ -130,6 +131,7 @@ void ExtensionGenerator::Generate(io::Printer* printer) { printer->Print(vars, "public static final int $constant_name$ = $number$;\n"); + WriteFieldDocComment(printer, descriptor_); if (HasDescriptorMethods(descriptor_->file())) { // Non-lite extensions if (descriptor_->extension_scope() == NULL) { diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_field.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_field.h index 6097f35..4dd0efd 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_field.h @@ -66,6 +66,7 @@ class FieldGenerator { virtual void GenerateBuildingCode(io::Printer* printer) const = 0; virtual void GenerateParsingCode(io::Printer* printer) const = 0; virtual void GenerateParsingCodeFromPacked(io::Printer* printer) const; + virtual void GenerateParsingDoneCode(io::Printer* printer) const = 0; virtual void GenerateSerializationCode(io::Printer* printer) const = 0; virtual void GenerateSerializedSizeCode(io::Printer* printer) const = 0; virtual void GenerateFieldBuilderInitializationCode(io::Printer* printer) diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_file.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_file.cc index 8968069..f43e550 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_file.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_file.cc @@ -42,6 +42,7 @@ #include <google/protobuf/io/printer.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/descriptor.pb.h> +#include <google/protobuf/dynamic_message.h> #include <google/protobuf/stubs/strutil.h> namespace google { @@ -51,18 +52,24 @@ namespace java { namespace { -// Recursively searches the given message to see if it contains any extensions. -bool UsesExtensions(const Message& message) { + +// Recursively searches the given message to collect extensions. +// Returns true if all the extensions can be recognized. The extensions will be +// appended in to the extensions parameter. +// Returns false when there are unknown fields, in which case the data in the +// extensions output parameter is not reliable and should be discarded. +bool CollectExtensions(const Message& message, + vector<const FieldDescriptor*>* extensions) { const Reflection* reflection = message.GetReflection(); - // We conservatively assume that unknown fields are extensions. - if (reflection->GetUnknownFields(message).field_count() > 0) return true; + // There are unknown fields that could be extensions, thus this call fails. + if (reflection->GetUnknownFields(message).field_count() > 0) return false; vector<const FieldDescriptor*> fields; reflection->ListFields(message, &fields); for (int i = 0; i < fields.size(); i++) { - if (fields[i]->is_extension()) return true; + if (fields[i]->is_extension()) extensions->push_back(fields[i]); if (GetJavaType(fields[i]) == JAVATYPE_MESSAGE) { if (fields[i]->is_repeated()) { @@ -70,16 +77,56 @@ bool UsesExtensions(const Message& message) { for (int j = 0; j < size; j++) { const Message& sub_message = reflection->GetRepeatedMessage(message, fields[i], j); - if (UsesExtensions(sub_message)) return true; + if (!CollectExtensions(sub_message, extensions)) return false; } } else { const Message& sub_message = reflection->GetMessage(message, fields[i]); - if (UsesExtensions(sub_message)) return true; + if (!CollectExtensions(sub_message, extensions)) return false; } } } - return false; + return true; +} + +// Finds all extensions in the given message and its sub-messages. If the +// message contains unknown fields (which could be extensions), then those +// extensions are defined in alternate_pool. +// The message will be converted to a DynamicMessage backed by alternate_pool +// in order to handle this case. +void CollectExtensions(const FileDescriptorProto& file_proto, + const DescriptorPool& alternate_pool, + vector<const FieldDescriptor*>* extensions, + const string& file_data) { + if (!CollectExtensions(file_proto, extensions)) { + // There are unknown fields in the file_proto, which are probably + // extensions. We need to parse the data into a dynamic message based on the + // builder-pool to find out all extensions. + const Descriptor* file_proto_desc = alternate_pool.FindMessageTypeByName( + file_proto.GetDescriptor()->full_name()); + GOOGLE_CHECK(file_proto_desc) + << "Find unknown fields in FileDescriptorProto when building " + << file_proto.name() + << ". It's likely that those fields are custom options, however, " + "descriptor.proto is not in the transitive dependencies. " + "This normally should not happen. Please report a bug."; + DynamicMessageFactory factory; + scoped_ptr<Message> dynamic_file_proto( + factory.GetPrototype(file_proto_desc)->New()); + GOOGLE_CHECK(dynamic_file_proto.get() != NULL); + GOOGLE_CHECK(dynamic_file_proto->ParseFromString(file_data)); + + // Collect the extensions again from the dynamic message. There should be no + // more unknown fields this time, i.e. all the custom options should be + // parsed as extensions now. + extensions->clear(); + GOOGLE_CHECK(CollectExtensions(*dynamic_file_proto, extensions)) + << "Find unknown fields in FileDescriptorProto when building " + << file_proto.name() + << ". It's likely that those fields are custom options, however, " + "those options cannot be recognized in the builder pool. " + "This normally should not happen. Please report a bug."; + } } @@ -306,19 +353,32 @@ void FileGenerator::GenerateEmbeddedDescriptor(io::Printer* printer) { .GenerateNonNestedInitializationCode(printer); } - if (UsesExtensions(file_proto)) { - // Must construct an ExtensionRegistry containing all possible extensions + // Proto compiler builds a DescriptorPool, which holds all the descriptors to + // generate, when processing the ".proto" files. We call this DescriptorPool + // the parsed pool (a.k.a. file_->pool()). + // + // Note that when users try to extend the (.*)DescriptorProto in their + // ".proto" files, it does not affect the pre-built FileDescriptorProto class + // in proto compiler. When we put the descriptor data in the file_proto, those + // extensions become unknown fields. + // + // Now we need to find out all the extension value to the (.*)DescriptorProto + // in the file_proto message, and prepare an ExtensionRegistry to return. + // + // To find those extensions, we need to parse the data into a dynamic message + // of the FileDescriptor based on the builder-pool, then we can use + // reflections to find all extension fields + vector<const FieldDescriptor*> extensions; + CollectExtensions(file_proto, *file_->pool(), &extensions, file_data); + + if (extensions.size() > 0) { + // Must construct an ExtensionRegistry containing all existing extensions // and return it. printer->Print( "com.google.protobuf.ExtensionRegistry registry =\n" - " com.google.protobuf.ExtensionRegistry.newInstance();\n" - "registerAllExtensions(registry);\n"); - for (int i = 0; i < file_->dependency_count(); i++) { - if (ShouldIncludeDependency(file_->dependency(i))) { - printer->Print( - "$dependency$.registerAllExtensions(registry);\n", - "dependency", ClassName(file_->dependency(i))); - } + " com.google.protobuf.ExtensionRegistry.newInstance();\n"); + for (int i = 0; i < extensions.size(); i++) { + ExtensionGenerator(extensions[i]).GenerateRegistrationCode(printer); } printer->Print( "return registry;\n"); @@ -375,7 +435,9 @@ static void GenerateSibling(const string& package_dir, printer.Print( "// Generated by the protocol buffer compiler. DO NOT EDIT!\n" - "\n"); + "// source: $filename$\n" + "\n", + "filename", descriptor->file()->name()); if (!java_package.empty()) { printer.Print( "package $package$;\n" diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.cc index 1b6f165..cf241b8 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.cc @@ -177,6 +177,18 @@ string ToJavaName(const string& full_name, const FileDescriptor* file) { return result; } +string ClassName(const Descriptor* descriptor) { + return ToJavaName(descriptor->full_name(), descriptor->file()); +} + +string ClassName(const EnumDescriptor* descriptor) { + return ToJavaName(descriptor->full_name(), descriptor->file()); +} + +string ClassName(const ServiceDescriptor* descriptor) { + return ToJavaName(descriptor->full_name(), descriptor->file()); +} + string ClassName(const FileDescriptor* descriptor) { string result = FileJavaPackage(descriptor); if (!result.empty()) result += '.'; @@ -326,14 +338,14 @@ string DefaultValue(const FieldDescriptor* field) { } else { // See comments in Internal.java for gory details. return strings::Substitute( - "com.google.protobuf.Internal.stringDefaultValue(\"$0\")", - CEscape(field->default_value_string())); + "com.google.protobuf.Internal.stringDefaultValue(\"$0\")", + CEscape(field->default_value_string())); } } case FieldDescriptor::CPPTYPE_ENUM: return ClassName(field->enum_type()) + "." + - field->default_value_enum()->name(); + field->default_value_enum()->name(); case FieldDescriptor::CPPTYPE_MESSAGE: return ClassName(field->message_type()) + ".getDefaultInstance()"; @@ -427,8 +439,10 @@ string GetBitFieldNameForBit(int bitIndex) { return GetBitFieldName(bitIndex / 32); } -string GenerateGetBit(int bitIndex) { - string varName = GetBitFieldNameForBit(bitIndex); +namespace { + +string GenerateGetBitInternal(const string& prefix, int bitIndex) { + string varName = prefix + GetBitFieldNameForBit(bitIndex); int bitInVarIndex = bitIndex % 32; string mask = bit_masks[bitInVarIndex]; @@ -436,8 +450,8 @@ string GenerateGetBit(int bitIndex) { return result; } -string GenerateSetBit(int bitIndex) { - string varName = GetBitFieldNameForBit(bitIndex); +string GenerateSetBitInternal(const string& prefix, int bitIndex) { + string varName = prefix + GetBitFieldNameForBit(bitIndex); int bitInVarIndex = bitIndex % 32; string mask = bit_masks[bitInVarIndex]; @@ -445,6 +459,16 @@ string GenerateSetBit(int bitIndex) { return result; } +} // namespace + +string GenerateGetBit(int bitIndex) { + return GenerateGetBitInternal("", bitIndex); +} + +string GenerateSetBit(int bitIndex) { + return GenerateSetBitInternal("", bitIndex); +} + string GenerateClearBit(int bitIndex) { string varName = GetBitFieldNameForBit(bitIndex); int bitInVarIndex = bitIndex % 32; @@ -455,21 +479,19 @@ string GenerateClearBit(int bitIndex) { } string GenerateGetBitFromLocal(int bitIndex) { - string varName = "from_" + GetBitFieldNameForBit(bitIndex); - int bitInVarIndex = bitIndex % 32; - - string mask = bit_masks[bitInVarIndex]; - string result = "((" + varName + " & " + mask + ") == " + mask + ")"; - return result; + return GenerateGetBitInternal("from_", bitIndex); } string GenerateSetBitToLocal(int bitIndex) { - string varName = "to_" + GetBitFieldNameForBit(bitIndex); - int bitInVarIndex = bitIndex % 32; + return GenerateSetBitInternal("to_", bitIndex); +} - string mask = bit_masks[bitInVarIndex]; - string result = varName + " |= " + mask; - return result; +string GenerateGetBitMutableLocal(int bitIndex) { + return GenerateGetBitInternal("mutable_", bitIndex); +} + +string GenerateSetBitMutableLocal(int bitIndex) { + return GenerateSetBitInternal("mutable_", bitIndex); } } // namespace java diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.h index 88efd45..3ef7149 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_helpers.h @@ -78,19 +78,14 @@ string ToJavaName(const string& full_name, const FileDescriptor* file); // These return the fully-qualified class name corresponding to the given // descriptor. -inline string ClassName(const Descriptor* descriptor) { - return ToJavaName(descriptor->full_name(), descriptor->file()); -} -inline string ClassName(const EnumDescriptor* descriptor) { - return ToJavaName(descriptor->full_name(), descriptor->file()); -} -inline string ClassName(const ServiceDescriptor* descriptor) { - return ToJavaName(descriptor->full_name(), descriptor->file()); -} +string ClassName(const Descriptor* descriptor); +string ClassName(const EnumDescriptor* descriptor); +string ClassName(const ServiceDescriptor* descriptor); +string ClassName(const FileDescriptor* descriptor); + inline string ExtensionIdentifierName(const FieldDescriptor* descriptor) { return ToJavaName(descriptor->full_name(), descriptor->file()); } -string ClassName(const FileDescriptor* descriptor); // Get the unqualified name that should be used for a field's field // number constant. @@ -206,6 +201,18 @@ string GenerateGetBitFromLocal(int bitIndex); // Example: "to_bitField1_ = (to_bitField1_ | 0x04)" string GenerateSetBitToLocal(int bitIndex); +// Does the same as GenerateGetBit but operates on the bit field on a local +// variable. This is used by the parsing constructor to record if a repeated +// field is mutable. +// Example: "((mutable_bitField1_ & 0x04) == 0x04)" +string GenerateGetBitMutableLocal(int bitIndex); + +// Does the same as GenerateSetBit but operates on the bit field on a local +// variable. This is used by the parsing constructor to record if a repeated +// field is mutable. +// Example: "mutable_bitField1_ = (mutable_bitField1_ | 0x04)" +string GenerateSetBitMutableLocal(int bitIndex); + } // namespace java } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_message.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_message.cc index 4c087db..9322e24 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_message.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_message.cc @@ -32,17 +32,23 @@ // Based on original Protocol Buffers design by // Sanjay Ghemawat, Jeff Dean, and others. +#include <google/protobuf/compiler/java/java_message.h> + #include <algorithm> #include <google/protobuf/stubs/hash.h> -#include <google/protobuf/compiler/java/java_message.h> +#include <map> +#include <vector> + +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/compiler/java/java_enum.h> #include <google/protobuf/compiler/java/java_extension.h> #include <google/protobuf/compiler/java/java_helpers.h> -#include <google/protobuf/stubs/strutil.h> -#include <google/protobuf/io/printer.h> #include <google/protobuf/io/coded_stream.h> -#include <google/protobuf/wire_format.h> +#include <google/protobuf/io/printer.h> #include <google/protobuf/descriptor.pb.h> +#include <google/protobuf/wire_format.h> +#include <google/protobuf/stubs/strutil.h> +#include <google/protobuf/stubs/substitute.h> namespace google { namespace protobuf { @@ -233,10 +239,8 @@ void MessageGenerator::GenerateStaticVariableInitializers( "field_name", UnderscoresToCapitalizedCamelCase(descriptor_->field(i))); } - printer->Print("},\n" - " $classname$.class,\n" - " $classname$.Builder.class);\n", - "classname", ClassName(descriptor_)); + printer->Print( + "});\n"); } // Generate static member initializers for all nested types. @@ -250,7 +254,6 @@ void MessageGenerator::GenerateStaticVariableInitializers( // =================================================================== void MessageGenerator::GenerateInterface(io::Printer* printer) { - if (descriptor_->extension_range_count() > 0) { if (HasDescriptorMethods(descriptor_)) { printer->Print( @@ -298,6 +301,10 @@ void MessageGenerator::Generate(io::Printer* printer) { descriptor_->containing_type() == NULL && descriptor_->file()->options().java_multiple_files(); + WriteMessageDocComment(printer, descriptor_); + + // The builder_type stores the super type name of the nested Builder class. + string builder_type; if (descriptor_->extension_range_count() > 0) { if (HasDescriptorMethods(descriptor_)) { printer->Print( @@ -306,6 +313,9 @@ void MessageGenerator::Generate(io::Printer* printer) { " $classname$> implements $classname$OrBuilder {\n", "static", is_own_file ? "" : "static", "classname", descriptor_->name()); + builder_type = strings::Substitute( + "com.google.protobuf.GeneratedMessage.ExtendableBuilder<$0, ?>", + ClassName(descriptor_)); } else { printer->Print( "public $static$ final class $classname$ extends\n" @@ -313,6 +323,9 @@ void MessageGenerator::Generate(io::Printer* printer) { " $classname$> implements $classname$OrBuilder {\n", "static", is_own_file ? "" : "static", "classname", descriptor_->name()); + builder_type = strings::Substitute( + "com.google.protobuf.GeneratedMessageLite.ExtendableBuilder<$0, ?>", + ClassName(descriptor_)); } } else { if (HasDescriptorMethods(descriptor_)) { @@ -322,6 +335,7 @@ void MessageGenerator::Generate(io::Printer* printer) { " implements $classname$OrBuilder {\n", "static", is_own_file ? "" : "static", "classname", descriptor_->name()); + builder_type = "com.google.protobuf.GeneratedMessage.Builder<?>"; } else { printer->Print( "public $static$ final class $classname$ extends\n" @@ -329,17 +343,28 @@ void MessageGenerator::Generate(io::Printer* printer) { " implements $classname$OrBuilder {\n", "static", is_own_file ? "" : "static", "classname", descriptor_->name()); + builder_type = "com.google.protobuf.GeneratedMessageLite.Builder"; } } printer->Indent(); + // Using builder_type, instead of Builder, prevents the Builder class from + // being loaded into PermGen space when the default instance is created. + // This optimizes the PermGen space usage for clients that do not modify + // messages. printer->Print( "// Use $classname$.newBuilder() to construct.\n" - "private $classname$(Builder builder) {\n" + "private $classname$($buildertype$ builder) {\n" " super(builder);\n" - "}\n" + "$set_unknown_fields$\n" + "}\n", + "classname", descriptor_->name(), + "buildertype", builder_type, + "set_unknown_fields", HasUnknownFields(descriptor_) + ? " this.unknownFields = builder.getUnknownFields();" : ""); + printer->Print( // Used when constructing the default instance, which cannot be initialized // immediately because it may cyclically refer to other default instances. - "private $classname$(boolean noInit) {}\n" + "private $classname$(boolean noInit) {$set_default_unknown_fields$}\n" "\n" "private static final $classname$ defaultInstance;\n" "public static $classname$ getDefaultInstance() {\n" @@ -350,9 +375,28 @@ void MessageGenerator::Generate(io::Printer* printer) { " return defaultInstance;\n" "}\n" "\n", - "classname", descriptor_->name()); + "classname", descriptor_->name(), + "set_default_unknown_fields", HasUnknownFields(descriptor_) + ? " this.unknownFields =" + " com.google.protobuf.UnknownFieldSet.getDefaultInstance(); " : ""); + + if (HasUnknownFields(descriptor_)) { + printer->Print( + "private final com.google.protobuf.UnknownFieldSet unknownFields;\n" + "" + "@java.lang.Override\n" + "public final com.google.protobuf.UnknownFieldSet\n" + " getUnknownFields() {\n" + " return this.unknownFields;\n" + "}\n"); + } + + if (HasGeneratedMethods(descriptor_)) { + GenerateParsingConstructor(printer); + } GenerateDescriptorMethods(printer); + GenerateParser(printer); // Nested types for (int i = 0; i < descriptor_->enum_type_count(); i++) { @@ -567,68 +611,54 @@ GenerateParseFromMethods(io::Printer* printer) { "public static $classname$ parseFrom(\n" " com.google.protobuf.ByteString data)\n" " throws com.google.protobuf.InvalidProtocolBufferException {\n" - " return newBuilder().mergeFrom(data).buildParsed();\n" + " return PARSER.parseFrom(data);\n" "}\n" "public static $classname$ parseFrom(\n" " com.google.protobuf.ByteString data,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" " throws com.google.protobuf.InvalidProtocolBufferException {\n" - " return newBuilder().mergeFrom(data, extensionRegistry)\n" - " .buildParsed();\n" + " return PARSER.parseFrom(data, extensionRegistry);\n" "}\n" "public static $classname$ parseFrom(byte[] data)\n" " throws com.google.protobuf.InvalidProtocolBufferException {\n" - " return newBuilder().mergeFrom(data).buildParsed();\n" + " return PARSER.parseFrom(data);\n" "}\n" "public static $classname$ parseFrom(\n" " byte[] data,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" " throws com.google.protobuf.InvalidProtocolBufferException {\n" - " return newBuilder().mergeFrom(data, extensionRegistry)\n" - " .buildParsed();\n" + " return PARSER.parseFrom(data, extensionRegistry);\n" "}\n" "public static $classname$ parseFrom(java.io.InputStream input)\n" " throws java.io.IOException {\n" - " return newBuilder().mergeFrom(input).buildParsed();\n" + " return PARSER.parseFrom(input);\n" "}\n" "public static $classname$ parseFrom(\n" " java.io.InputStream input,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" " throws java.io.IOException {\n" - " return newBuilder().mergeFrom(input, extensionRegistry)\n" - " .buildParsed();\n" + " return PARSER.parseFrom(input, extensionRegistry);\n" "}\n" "public static $classname$ parseDelimitedFrom(java.io.InputStream input)\n" " throws java.io.IOException {\n" - " Builder builder = newBuilder();\n" - " if (builder.mergeDelimitedFrom(input)) {\n" - " return builder.buildParsed();\n" - " } else {\n" - " return null;\n" - " }\n" + " return PARSER.parseDelimitedFrom(input);\n" "}\n" "public static $classname$ parseDelimitedFrom(\n" " java.io.InputStream input,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" " throws java.io.IOException {\n" - " Builder builder = newBuilder();\n" - " if (builder.mergeDelimitedFrom(input, extensionRegistry)) {\n" - " return builder.buildParsed();\n" - " } else {\n" - " return null;\n" - " }\n" + " return PARSER.parseDelimitedFrom(input, extensionRegistry);\n" "}\n" "public static $classname$ parseFrom(\n" " com.google.protobuf.CodedInputStream input)\n" " throws java.io.IOException {\n" - " return newBuilder().mergeFrom(input).buildParsed();\n" + " return PARSER.parseFrom(input);\n" "}\n" "public static $classname$ parseFrom(\n" " com.google.protobuf.CodedInputStream input,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" " throws java.io.IOException {\n" - " return newBuilder().mergeFrom(input, extensionRegistry)\n" - " .buildParsed();\n" + " return PARSER.parseFrom(input, extensionRegistry);\n" "}\n" "\n", "classname", ClassName(descriptor_)); @@ -669,6 +699,8 @@ void MessageGenerator::GenerateBuilder(io::Printer* printer) { "}\n"); } + WriteMessageDocComment(printer, descriptor_); + if (descriptor_->extension_range_count() > 0) { if (HasDescriptorMethods(descriptor_)) { printer->Print( @@ -739,17 +771,25 @@ void MessageGenerator::GenerateBuilder(io::Printer* printer) { void MessageGenerator::GenerateDescriptorMethods(io::Printer* printer) { if (HasDescriptorMethods(descriptor_)) { + if (!descriptor_->options().no_standard_descriptor_accessor()) { + printer->Print( + "public static final com.google.protobuf.Descriptors.Descriptor\n" + " getDescriptor() {\n" + " return $fileclass$.internal_$identifier$_descriptor;\n" + "}\n" + "\n", + "fileclass", ClassName(descriptor_->file()), + "identifier", UniqueFileScopeIdentifier(descriptor_)); + } printer->Print( - "public static final com.google.protobuf.Descriptors.Descriptor\n" - " getDescriptor() {\n" - " return $fileclass$.internal_$identifier$_descriptor;\n" - "}\n" - "\n" "protected com.google.protobuf.GeneratedMessage.FieldAccessorTable\n" " internalGetFieldAccessorTable() {\n" - " return $fileclass$.internal_$identifier$_fieldAccessorTable;\n" + " return $fileclass$.internal_$identifier$_fieldAccessorTable\n" + " .ensureFieldAccessorsInitialized(\n" + " $classname$.class, $classname$.Builder.class);\n" "}\n" "\n", + "classname", ClassName(descriptor_), "fileclass", ClassName(descriptor_->file()), "identifier", UniqueFileScopeIdentifier(descriptor_)); } @@ -768,7 +808,8 @@ void MessageGenerator::GenerateCommonBuilderMethods(io::Printer* printer) { if (HasDescriptorMethods(descriptor_)) { printer->Print( - "private Builder(BuilderParent parent) {\n" + "private Builder(\n" + " com.google.protobuf.GeneratedMessage.BuilderParent parent) {\n" " super(parent);\n" " maybeForceBuilderInitialization();\n" "}\n", @@ -830,10 +871,11 @@ void MessageGenerator::GenerateCommonBuilderMethods(io::Printer* printer) { printer->Print( "public com.google.protobuf.Descriptors.Descriptor\n" " getDescriptorForType() {\n" - " return $classname$.getDescriptor();\n" + " return $fileclass$.internal_$identifier$_descriptor;\n" "}\n" "\n", - "classname", ClassName(descriptor_)); + "fileclass", ClassName(descriptor_->file()), + "identifier", UniqueFileScopeIdentifier(descriptor_)); } printer->Print( "public $classname$ getDefaultInstanceForType() {\n" @@ -853,16 +895,6 @@ void MessageGenerator::GenerateCommonBuilderMethods(io::Printer* printer) { " return result;\n" "}\n" "\n" - "private $classname$ buildParsed()\n" - " throws com.google.protobuf.InvalidProtocolBufferException {\n" - " $classname$ result = buildPartial();\n" - " if (!result.isInitialized()) {\n" - " throw newUninitializedMessageException(\n" - " result).asInvalidProtocolBufferException();\n" - " }\n" - " return result;\n" - "}\n" - "\n" "public $classname$ buildPartial() {\n" " $classname$ result = new $classname$(this);\n", "classname", ClassName(descriptor_)); @@ -969,108 +1001,25 @@ void MessageGenerator::GenerateCommonBuilderMethods(io::Printer* printer) { // =================================================================== void MessageGenerator::GenerateBuilderParsingMethods(io::Printer* printer) { - scoped_array<const FieldDescriptor*> sorted_fields( - SortFieldsByNumber(descriptor_)); - printer->Print( "public Builder mergeFrom(\n" " com.google.protobuf.CodedInputStream input,\n" " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" - " throws java.io.IOException {\n"); - printer->Indent(); - - if (HasUnknownFields(descriptor_)) { - printer->Print( - "com.google.protobuf.UnknownFieldSet.Builder unknownFields =\n" - " com.google.protobuf.UnknownFieldSet.newBuilder(\n" - " this.getUnknownFields());\n"); - } - - printer->Print( - "while (true) {\n"); - printer->Indent(); - - printer->Print( - "int tag = input.readTag();\n" - "switch (tag) {\n"); - printer->Indent(); - - if (HasUnknownFields(descriptor_)) { - printer->Print( - "case 0:\n" // zero signals EOF / limit reached - " this.setUnknownFields(unknownFields.build());\n" - " $on_changed$\n" - " return this;\n" - "default: {\n" - " if (!parseUnknownField(input, unknownFields,\n" - " extensionRegistry, tag)) {\n" - " this.setUnknownFields(unknownFields.build());\n" - " $on_changed$\n" - " return this;\n" // it's an endgroup tag - " }\n" - " break;\n" - "}\n", - "on_changed", HasDescriptorMethods(descriptor_) ? "onChanged();" : ""); - } else { - printer->Print( - "case 0:\n" // zero signals EOF / limit reached - " $on_changed$\n" - " return this;\n" - "default: {\n" - " if (!parseUnknownField(input, extensionRegistry, tag)) {\n" - " $on_changed$\n" - " return this;\n" // it's an endgroup tag - " }\n" - " break;\n" - "}\n", - "on_changed", HasDescriptorMethods(descriptor_) ? "onChanged();" : ""); - } - - for (int i = 0; i < descriptor_->field_count(); i++) { - const FieldDescriptor* field = sorted_fields[i]; - uint32 tag = WireFormatLite::MakeTag(field->number(), - WireFormat::WireTypeForFieldType(field->type())); - - printer->Print( - "case $tag$: {\n", - "tag", SimpleItoa(tag)); - printer->Indent(); - - field_generators_.get(field).GenerateParsingCode(printer); - - printer->Outdent(); - printer->Print( - " break;\n" - "}\n"); - - if (field->is_packable()) { - // To make packed = true wire compatible, we generate parsing code from a - // packed version of this field regardless of field->options().packed(). - uint32 packed_tag = WireFormatLite::MakeTag(field->number(), - WireFormatLite::WIRETYPE_LENGTH_DELIMITED); - printer->Print( - "case $tag$: {\n", - "tag", SimpleItoa(packed_tag)); - printer->Indent(); - - field_generators_.get(field).GenerateParsingCodeFromPacked(printer); - - printer->Outdent(); - printer->Print( - " break;\n" - "}\n"); - } - } - - printer->Outdent(); - printer->Outdent(); - printer->Outdent(); - printer->Print( - " }\n" // switch (tag) - " }\n" // while (true) - "}\n" - - "\n"); + " throws java.io.IOException {\n" + " $classname$ parsedMessage = null;\n" + " try {\n" + " parsedMessage = PARSER.parsePartialFrom(input, extensionRegistry);\n" + " } catch (com.google.protobuf.InvalidProtocolBufferException e) {\n" + " parsedMessage = ($classname$) e.getUnfinishedMessage();\n" + " throw e;\n" + " } finally {\n" + " if (parsedMessage != null) {\n" + " mergeFrom(parsedMessage);\n" + " }\n" + " }\n" + " return this;\n" + "}\n", + "classname", ClassName(descriptor_)); } // =================================================================== @@ -1232,10 +1181,19 @@ void MessageGenerator::GenerateEqualsAndHashCode(io::Printer* printer) { "\n"); printer->Print( + "private int memoizedHashCode = 0;\n"); + printer->Print( "@java.lang.Override\n" "public int hashCode() {\n"); printer->Indent(); printer->Print( + "if (memoizedHashCode != 0) {\n"); + printer->Indent(); + printer->Print( + "return memoizedHashCode;\n"); + printer->Outdent(); + printer->Print( + "}\n" "int hash = 41;\n" "hash = (19 * hash) + getDescriptorForType().hashCode();\n"); for (int i = 0; i < descriptor_->field_count(); i++) { @@ -1260,6 +1218,7 @@ void MessageGenerator::GenerateEqualsAndHashCode(io::Printer* printer) { } printer->Print( "hash = (29 * hash) + getUnknownFields().hashCode();\n" + "memoizedHashCode = hash;\n" "return hash;\n"); printer->Outdent(); printer->Print( @@ -1281,6 +1240,195 @@ void MessageGenerator::GenerateExtensionRegistrationCode(io::Printer* printer) { } } +// =================================================================== +void MessageGenerator::GenerateParsingConstructor(io::Printer* printer) { + scoped_array<const FieldDescriptor*> sorted_fields( + SortFieldsByNumber(descriptor_)); + + printer->Print( + "private $classname$(\n" + " com.google.protobuf.CodedInputStream input,\n" + " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" + " throws com.google.protobuf.InvalidProtocolBufferException {\n", + "classname", descriptor_->name()); + printer->Indent(); + + // Initialize all fields to default. + printer->Print( + "initFields();\n"); + + // Use builder bits to track mutable repeated fields. + int totalBuilderBits = 0; + for (int i = 0; i < descriptor_->field_count(); i++) { + const FieldGenerator& field = field_generators_.get(descriptor_->field(i)); + totalBuilderBits += field.GetNumBitsForBuilder(); + } + int totalBuilderInts = (totalBuilderBits + 31) / 32; + for (int i = 0; i < totalBuilderInts; i++) { + printer->Print("int mutable_$bit_field_name$ = 0;\n", + "bit_field_name", GetBitFieldName(i)); + } + + if (HasUnknownFields(descriptor_)) { + printer->Print( + "com.google.protobuf.UnknownFieldSet.Builder unknownFields =\n" + " com.google.protobuf.UnknownFieldSet.newBuilder();\n"); + } + + printer->Print( + "try {\n"); + printer->Indent(); + + printer->Print( + "boolean done = false;\n" + "while (!done) {\n"); + printer->Indent(); + + printer->Print( + "int tag = input.readTag();\n" + "switch (tag) {\n"); + printer->Indent(); + + printer->Print( + "case 0:\n" // zero signals EOF / limit reached + " done = true;\n" + " break;\n" + "default: {\n" + " if (!parseUnknownField(input,$unknown_fields$\n" + " extensionRegistry, tag)) {\n" + " done = true;\n" // it's an endgroup tag + " }\n" + " break;\n" + "}\n", + "unknown_fields", HasUnknownFields(descriptor_) + ? " unknownFields," : ""); + + for (int i = 0; i < descriptor_->field_count(); i++) { + const FieldDescriptor* field = sorted_fields[i]; + uint32 tag = WireFormatLite::MakeTag(field->number(), + WireFormat::WireTypeForFieldType(field->type())); + + printer->Print( + "case $tag$: {\n", + "tag", SimpleItoa(tag)); + printer->Indent(); + + field_generators_.get(field).GenerateParsingCode(printer); + + printer->Outdent(); + printer->Print( + " break;\n" + "}\n"); + + if (field->is_packable()) { + // To make packed = true wire compatible, we generate parsing code from a + // packed version of this field regardless of field->options().packed(). + uint32 packed_tag = WireFormatLite::MakeTag(field->number(), + WireFormatLite::WIRETYPE_LENGTH_DELIMITED); + printer->Print( + "case $tag$: {\n", + "tag", SimpleItoa(packed_tag)); + printer->Indent(); + + field_generators_.get(field).GenerateParsingCodeFromPacked(printer); + + printer->Outdent(); + printer->Print( + " break;\n" + "}\n"); + } + } + + printer->Outdent(); + printer->Outdent(); + printer->Print( + " }\n" // switch (tag) + "}\n"); // while (!done) + + printer->Outdent(); + printer->Print( + "} catch (com.google.protobuf.InvalidProtocolBufferException e) {\n" + " throw e.setUnfinishedMessage(this);\n" + "} catch (java.io.IOException e) {\n" + " throw new com.google.protobuf.InvalidProtocolBufferException(\n" + " e.getMessage()).setUnfinishedMessage(this);\n" + "} finally {\n"); + printer->Indent(); + + // Make repeated field list immutable. + for (int i = 0; i < descriptor_->field_count(); i++) { + const FieldDescriptor* field = sorted_fields[i]; + field_generators_.get(field).GenerateParsingDoneCode(printer); + } + + // Make unknown fields immutable. + if (HasUnknownFields(descriptor_)) { + printer->Print( + "this.unknownFields = unknownFields.build();\n"); + } + + // Make extensions immutable. + printer->Print( + "makeExtensionsImmutable();\n"); + + printer->Outdent(); + printer->Outdent(); + printer->Print( + " }\n" // finally + "}\n"); +} + +// =================================================================== +void MessageGenerator::GenerateParser(io::Printer* printer) { + printer->Print( + "public static com.google.protobuf.Parser<$classname$> PARSER =\n" + " new com.google.protobuf.AbstractParser<$classname$>() {\n", + "classname", descriptor_->name()); + printer->Indent(); + printer->Print( + "public $classname$ parsePartialFrom(\n" + " com.google.protobuf.CodedInputStream input,\n" + " com.google.protobuf.ExtensionRegistryLite extensionRegistry)\n" + " throws com.google.protobuf.InvalidProtocolBufferException {\n", + "classname", descriptor_->name()); + if (HasGeneratedMethods(descriptor_)) { + printer->Print( + " return new $classname$(input, extensionRegistry);\n", + "classname", descriptor_->name()); + } else { + // When parsing constructor isn't generated, use builder to parse messages. + // Note, will fallback to use reflection based mergeFieldFrom() in + // AbstractMessage.Builder. + printer->Indent(); + printer->Print( + "Builder builder = newBuilder();\n" + "try {\n" + " builder.mergeFrom(input, extensionRegistry);\n" + "} catch (com.google.protobuf.InvalidProtocolBufferException e) {\n" + " throw e.setUnfinishedMessage(builder.buildPartial());\n" + "} catch (java.io.IOException e) {\n" + " throw new com.google.protobuf.InvalidProtocolBufferException(\n" + " e.getMessage()).setUnfinishedMessage(builder.buildPartial());\n" + "}\n" + "return builder.buildPartial();\n"); + printer->Outdent(); + } + printer->Print( + "}\n"); + printer->Outdent(); + printer->Print( + "};\n" + "\n"); + + printer->Print( + "@java.lang.Override\n" + "public com.google.protobuf.Parser<$classname$> getParserForType() {\n" + " return PARSER;\n" + "}\n" + "\n", + "classname", descriptor_->name()); +} + } // namespace java } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_message.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_message.h index 4c6fbbe..a30f020 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_message.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_message.h @@ -95,6 +95,9 @@ class MessageGenerator { UseMemoization useMemoization); void GenerateEqualsAndHashCode(io::Printer* printer); + void GenerateParser(io::Printer* printer); + void GenerateParsingConstructor(io::Printer* printer); + const Descriptor* descriptor_; FieldGeneratorMap field_generators_; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.cc index 251945a..b0b284f 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.cc @@ -36,6 +36,7 @@ #include <string> #include <google/protobuf/compiler/java/java_message_field.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/wire_format.h> @@ -73,6 +74,7 @@ void SetMessageVariables(const FieldDescriptor* descriptor, // For singular messages and builders, one bit is used for the hasField bit. (*variables)["get_has_field_bit_message"] = GenerateGetBit(messageBitIndex); + (*variables)["set_has_field_bit_message"] = GenerateSetBit(messageBitIndex); (*variables)["get_has_field_bit_builder"] = GenerateGetBit(builderBitIndex); (*variables)["set_has_field_bit_builder"] = GenerateSetBit(builderBitIndex); @@ -84,6 +86,13 @@ void SetMessageVariables(const FieldDescriptor* descriptor, (*variables)["set_mutable_bit_builder"] = GenerateSetBit(builderBitIndex); (*variables)["clear_mutable_bit_builder"] = GenerateClearBit(builderBitIndex); + // For repeated fields, one bit is used for whether the array is immutable + // in the parsing constructor. + (*variables)["get_mutable_bit_parser"] = + GenerateGetBitMutableLocal(builderBitIndex); + (*variables)["set_mutable_bit_parser"] = + GenerateSetBitMutableLocal(builderBitIndex); + (*variables)["get_has_field_bit_from_local"] = GenerateGetBitFromLocal(builderBitIndex); (*variables)["set_has_field_bit_to_local"] = @@ -120,11 +129,15 @@ GenerateInterfaceMembers(io::Printer* printer) const { // interface so that builders can choose dynamically to either return a // message or a nested builder, so that asking for the interface doesn't // cause a message to ever be built. + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$boolean has$capitalized_name$();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$boolean has$capitalized_name$();\n" "$deprecation$$type$ get$capitalized_name$();\n"); if (HasNestedBuilders(descriptor_->containing_type())) { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$$type$OrBuilder get$capitalized_name$OrBuilder();\n"); } @@ -133,15 +146,20 @@ GenerateInterfaceMembers(io::Printer* printer) const { void MessageFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private $type$ $name$_;\n" + "private $type$ $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_message$;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$() {\n" " return $name$_;\n" "}\n"); if (HasNestedBuilders(descriptor_->containing_type())) { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public $type$OrBuilder get$capitalized_name$OrBuilder() {\n" " return $name$_;\n" @@ -208,12 +226,14 @@ GenerateBuilderMembers(io::Printer* printer) const { // field of type "Field" called "Field". // boolean hasField() + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_builder$;\n" "}\n"); // Field getField() + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public $type$ get$capitalized_name$()", @@ -224,6 +244,7 @@ GenerateBuilderMembers(io::Printer* printer) const { NULL); // Field.Builder setField(Field value) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder set$capitalized_name$($type$ value)", @@ -239,6 +260,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Field.Builder setField(Field.Builder builderForValue) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder set$capitalized_name$(\n" " $type$.Builder builderForValue)", @@ -252,6 +274,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Field.Builder mergeField(Field value) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder merge$capitalized_name$($type$ value)", @@ -270,6 +293,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Field.Builder clearField() + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder clear$capitalized_name$()", @@ -282,19 +306,24 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); if (HasNestedBuilders(descriptor_->containing_type())) { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public $type$.Builder get$capitalized_name$Builder() {\n" " $set_has_field_bit_builder$;\n" " $on_changed$\n" " return get$capitalized_name$FieldBuilder().getBuilder();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$OrBuilder get$capitalized_name$OrBuilder() {\n" " if ($name$Builder_ != null) {\n" " return $name$Builder_.getMessageOrBuilder();\n" " } else {\n" " return $name$_;\n" " }\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "private com.google.protobuf.SingleFieldBuilder<\n" " $type$, $type$.Builder, $type$OrBuilder> \n" " get$capitalized_name$FieldBuilder() {\n" @@ -357,21 +386,32 @@ GenerateBuildingCode(io::Printer* printer) const { void MessageFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "$type$.Builder subBuilder = $type$.newBuilder();\n" - "if (has$capitalized_name$()) {\n" - " subBuilder.mergeFrom(get$capitalized_name$());\n" + "$type$.Builder subBuilder = null;\n" + "if ($get_has_field_bit_message$) {\n" + " subBuilder = $name$_.toBuilder();\n" "}\n"); if (GetType(descriptor_) == FieldDescriptor::TYPE_GROUP) { printer->Print(variables_, - "input.readGroup($number$, subBuilder, extensionRegistry);\n"); + "$name$_ = input.readGroup($number$, $type$.PARSER,\n" + " extensionRegistry);\n"); } else { printer->Print(variables_, - "input.readMessage(subBuilder, extensionRegistry);\n"); + "$name$_ = input.readMessage($type$.PARSER, extensionRegistry);\n"); } printer->Print(variables_, - "set$capitalized_name$(subBuilder.buildPartial());\n"); + "if (subBuilder != null) {\n" + " subBuilder.mergeFrom($name$_);\n" + " $name$_ = subBuilder.buildPartial();\n" + "}\n"); + printer->Print(variables_, + "$set_has_field_bit_message$;\n"); +} + +void MessageFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + // noop for messages. } void MessageFieldGenerator:: @@ -437,15 +477,23 @@ GenerateInterfaceMembers(io::Printer* printer) const { // interface so that builders can choose dynamically to either return a // message or a nested builder, so that asking for the interface doesn't // cause a message to ever be built. + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$java.util.List<$type$> \n" - " get$capitalized_name$List();\n" - "$deprecation$$type$ get$capitalized_name$(int index);\n" + " get$capitalized_name$List();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$$type$ get$capitalized_name$(int index);\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$int get$capitalized_name$Count();\n"); if (HasNestedBuilders(descriptor_->containing_type())) { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$java.util.List<? extends $type$OrBuilder> \n" - " get$capitalized_name$OrBuilderList();\n" + " get$capitalized_name$OrBuilderList();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$$type$OrBuilder get$capitalized_name$OrBuilder(\n" " int index);\n"); } @@ -454,20 +502,30 @@ GenerateInterfaceMembers(io::Printer* printer) const { void RepeatedMessageFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private java.util.List<$type$> $name$_;\n" + "private java.util.List<$type$> $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<$type$> get$capitalized_name$List() {\n" " return $name$_;\n" // note: unmodifiable list - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<? extends $type$OrBuilder> \n" " get$capitalized_name$OrBuilderList() {\n" " return $name$_;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$OrBuilder get$capitalized_name$OrBuilder(\n" " int index) {\n" " return $name$_.get(index);\n" @@ -552,6 +610,7 @@ GenerateBuilderMembers(io::Printer* printer) const { // repeated field of type "Field" called "RepeatedField". // List<Field> getRepeatedFieldList() + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public java.util.List<$type$> get$capitalized_name$List()", @@ -561,6 +620,7 @@ GenerateBuilderMembers(io::Printer* printer) const { NULL); // int getRepeatedFieldCount() + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public int get$capitalized_name$Count()", @@ -570,6 +630,7 @@ GenerateBuilderMembers(io::Printer* printer) const { NULL); // Field getRepeatedField(int index) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public $type$ get$capitalized_name$(int index)", @@ -580,6 +641,7 @@ GenerateBuilderMembers(io::Printer* printer) const { NULL); // Builder setRepeatedField(int index, Field value) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder set$capitalized_name$(\n" " int index, $type$ value)", @@ -593,6 +655,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder setRepeatedField(int index, Field.Builder builderForValue) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder set$capitalized_name$(\n" " int index, $type$.Builder builderForValue)", @@ -606,6 +669,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder addRepeatedField(Field value) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder add$capitalized_name$($type$ value)", @@ -622,6 +686,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder addRepeatedField(int index, Field value) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder add$capitalized_name$(\n" " int index, $type$ value)", @@ -638,6 +703,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder addRepeatedField(Field.Builder builderForValue) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder add$capitalized_name$(\n" " $type$.Builder builderForValue)", @@ -651,6 +717,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder addRepeatedField(int index, Field.Builder builderForValue) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder add$capitalized_name$(\n" " int index, $type$.Builder builderForValue)", @@ -664,6 +731,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder addAllRepeatedField(Iterable<Field> values) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder addAll$capitalized_name$(\n" " java.lang.Iterable<? extends $type$> values)", @@ -677,6 +745,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder clearAllRepeatedField() + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder clear$capitalized_name$()", @@ -689,6 +758,7 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); // Builder removeRepeatedField(int index) + WriteFieldDocComment(printer, descriptor_); PrintNestedBuilderFunction(printer, "$deprecation$public Builder remove$capitalized_name$(int index)", @@ -701,12 +771,15 @@ GenerateBuilderMembers(io::Printer* printer) const { "return this;\n"); if (HasNestedBuilders(descriptor_->containing_type())) { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public $type$.Builder get$capitalized_name$Builder(\n" " int index) {\n" " return get$capitalized_name$FieldBuilder().getBuilder(index);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$OrBuilder get$capitalized_name$OrBuilder(\n" " int index) {\n" " if ($name$Builder_ == null) {\n" @@ -714,8 +787,10 @@ GenerateBuilderMembers(io::Printer* printer) const { " } else {\n" " return $name$Builder_.getMessageOrBuilder(index);\n" " }\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<? extends $type$OrBuilder> \n" " get$capitalized_name$OrBuilderList() {\n" " if ($name$Builder_ != null) {\n" @@ -723,17 +798,23 @@ GenerateBuilderMembers(io::Printer* printer) const { " } else {\n" " return java.util.Collections.unmodifiableList($name$_);\n" " }\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$.Builder add$capitalized_name$Builder() {\n" " return get$capitalized_name$FieldBuilder().addBuilder(\n" " $type$.getDefaultInstance());\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$.Builder add$capitalized_name$Builder(\n" " int index) {\n" " return get$capitalized_name$FieldBuilder().addBuilder(\n" " index, $type$.getDefaultInstance());\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<$type$.Builder> \n" " get$capitalized_name$BuilderList() {\n" " return get$capitalized_name$FieldBuilder().getBuilderList();\n" @@ -827,18 +908,27 @@ GenerateBuildingCode(io::Printer* printer) const { void RepeatedMessageFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "$type$.Builder subBuilder = $type$.newBuilder();\n"); + "if (!$get_mutable_bit_parser$) {\n" + " $name$_ = new java.util.ArrayList<$type$>();\n" + " $set_mutable_bit_parser$;\n" + "}\n"); if (GetType(descriptor_) == FieldDescriptor::TYPE_GROUP) { printer->Print(variables_, - "input.readGroup($number$, subBuilder, extensionRegistry);\n"); + "$name$_.add(input.readGroup($number$, $type$.PARSER,\n" + " extensionRegistry));\n"); } else { printer->Print(variables_, - "input.readMessage(subBuilder, extensionRegistry);\n"); + "$name$_.add(input.readMessage($type$.PARSER, extensionRegistry));\n"); } +} +void RepeatedMessageFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { printer->Print(variables_, - "add$capitalized_name$(subBuilder.buildPartial());\n"); + "if ($get_mutable_bit_parser$) {\n" + " $name$_ = java.util.Collections.unmodifiableList($name$_);\n" + "}\n"); } void RepeatedMessageFieldGenerator:: diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.h index 2efbcd9..5c8078a 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_message_field.h @@ -61,6 +61,7 @@ class MessageFieldGenerator : public FieldGenerator { void GenerateMergingCode(io::Printer* printer) const; void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; @@ -102,6 +103,7 @@ class RepeatedMessageFieldGenerator : public FieldGenerator { void GenerateMergingCode(io::Printer* printer) const; void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.cc index 712e047..0140e23 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.cc @@ -36,6 +36,7 @@ #include <string> #include <google/protobuf/compiler/java/java_primitive_field.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/stubs/common.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> @@ -197,6 +198,7 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, // For singular messages and builders, one bit is used for the hasField bit. (*variables)["get_has_field_bit_message"] = GenerateGetBit(messageBitIndex); + (*variables)["set_has_field_bit_message"] = GenerateSetBit(messageBitIndex); (*variables)["get_has_field_bit_builder"] = GenerateGetBit(builderBitIndex); (*variables)["set_has_field_bit_builder"] = GenerateSetBit(builderBitIndex); @@ -208,6 +210,13 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, (*variables)["set_mutable_bit_builder"] = GenerateSetBit(builderBitIndex); (*variables)["clear_mutable_bit_builder"] = GenerateClearBit(builderBitIndex); + // For repeated fields, one bit is used for whether the array is immutable + // in the parsing constructor. + (*variables)["get_mutable_bit_parser"] = + GenerateGetBitMutableLocal(builderBitIndex); + (*variables)["set_mutable_bit_parser"] = + GenerateSetBitMutableLocal(builderBitIndex); + (*variables)["get_has_field_bit_from_local"] = GenerateGetBitFromLocal(builderBitIndex); (*variables)["set_has_field_bit_to_local"] = @@ -240,19 +249,26 @@ int PrimitiveFieldGenerator::GetNumBitsForBuilder() const { void PrimitiveFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$boolean has$capitalized_name$();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$boolean has$capitalized_name$();\n" "$deprecation$$type$ get$capitalized_name$();\n"); } void PrimitiveFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private $field_type$ $name$_;\n" + "private $field_type$ $name$_;\n"); + + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_message$;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$() {\n" " return $name$_;\n" @@ -262,16 +278,21 @@ GenerateMembers(io::Printer* printer) const { void PrimitiveFieldGenerator:: GenerateBuilderMembers(io::Printer* printer) const { printer->Print(variables_, - "private $field_type$ $name$_ $default_init$;\n" + "private $field_type$ $name$_ $default_init$;\n"); + + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_builder$;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$() {\n" " return $name$_;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$($type$ value) {\n" "$null_check$" @@ -279,7 +300,10 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_ = value;\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $clear_has_field_bit_builder$;\n"); JavaType type = GetJavaType(descriptor_); @@ -335,11 +359,16 @@ GenerateBuildingCode(io::Printer* printer) const { void PrimitiveFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "$set_has_field_bit_builder$;\n" + "$set_has_field_bit_message$;\n" "$name$_ = input.read$capitalized_type$();\n"); } void PrimitiveFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + // noop for primitives. +} + +void PrimitiveFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { printer->Print(variables_, "if ($get_has_field_bit_message$) {\n" @@ -468,9 +497,14 @@ int RepeatedPrimitiveFieldGenerator::GetNumBitsForBuilder() const { void RepeatedPrimitiveFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$java.util.List<$boxed_type$> get$capitalized_name$List();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$int get$capitalized_name$Count();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$java.util.List<$boxed_type$> get$capitalized_name$List();\n" - "$deprecation$int get$capitalized_name$Count();\n" "$deprecation$$type$ get$capitalized_name$(int index);\n"); } @@ -478,14 +512,20 @@ GenerateInterfaceMembers(io::Printer* printer) const { void RepeatedPrimitiveFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private $field_list_type$ $name$_;\n" + "private $field_list_type$ $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<$boxed_type$>\n" " get$capitalized_name$List() {\n" " return $name$_;\n" // note: unmodifiable list - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" "}\n"); @@ -523,17 +563,24 @@ GenerateBuilderMembers(io::Printer* printer) const { // could hold on to the returned list and modify it after the message // has been built, thus mutating the message which is supposed to be // immutable. + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public java.util.List<$boxed_type$>\n" " get$capitalized_name$List() {\n" " return java.util.Collections.unmodifiableList($name$_);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public $type$ get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$(\n" " int index, $type$ value) {\n" "$null_check$" @@ -541,21 +588,27 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_.set(index, value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder add$capitalized_name$($type$ value) {\n" "$null_check$" " ensure$capitalized_name$IsMutable();\n" " $name$_.add(value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder addAll$capitalized_name$(\n" " java.lang.Iterable<? extends $boxed_type$> values) {\n" " ensure$capitalized_name$IsMutable();\n" " super.addAll(values, $name$_);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $name$_ = $empty_list$;\n" " $clear_mutable_bit_builder$;\n" @@ -616,7 +669,10 @@ GenerateBuildingCode(io::Printer* printer) const { void RepeatedPrimitiveFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "ensure$capitalized_name$IsMutable();\n" + "if (!$get_mutable_bit_parser$) {\n" + " $name$_ = new java.util.ArrayList<$boxed_type$>();\n" + " $set_mutable_bit_parser$;\n" + "}\n" "$name$_.add(input.read$capitalized_type$());\n"); } @@ -625,13 +681,25 @@ GenerateParsingCodeFromPacked(io::Printer* printer) const { printer->Print(variables_, "int length = input.readRawVarint32();\n" "int limit = input.pushLimit(length);\n" + "if (!$get_mutable_bit_parser$ && input.getBytesUntilLimit() > 0) {\n" + " $name$_ = new java.util.ArrayList<$boxed_type$>();\n" + " $set_mutable_bit_parser$;\n" + "}\n" "while (input.getBytesUntilLimit() > 0) {\n" - " add$capitalized_name$(input.read$capitalized_type$());\n" + " $name$_.add(input.read$capitalized_type$());\n" "}\n" "input.popLimit(limit);\n"); } void RepeatedPrimitiveFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + printer->Print(variables_, + "if ($get_mutable_bit_parser$) {\n" + " $name$_ = java.util.Collections.unmodifiableList($name$_);\n" + "}\n"); +} + +void RepeatedPrimitiveFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { if (descriptor_->options().packed()) { printer->Print(variables_, diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.h index 7900fac..1b5b6d9 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_primitive_field.h @@ -61,6 +61,7 @@ class PrimitiveFieldGenerator : public FieldGenerator { void GenerateMergingCode(io::Printer* printer) const; void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; @@ -96,6 +97,7 @@ class RepeatedPrimitiveFieldGenerator : public FieldGenerator { void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; void GenerateParsingCodeFromPacked(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_service.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_service.cc index 2d85b7a..bcd80359 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_service.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_service.cc @@ -33,6 +33,7 @@ // Sanjay Ghemawat, Jeff Dean, and others. #include <google/protobuf/compiler/java/java_service.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/descriptor.pb.h> @@ -50,6 +51,7 @@ ServiceGenerator::~ServiceGenerator() {} void ServiceGenerator::Generate(io::Printer* printer) { bool is_own_file = descriptor_->file()->options().java_multiple_files(); + WriteServiceDocComment(printer, descriptor_); printer->Print( "public $static$ abstract class $classname$\n" " implements com.google.protobuf.Service {\n", @@ -163,6 +165,7 @@ void ServiceGenerator::GenerateNewReflectiveBlockingServiceMethod( void ServiceGenerator::GenerateAbstractMethods(io::Printer* printer) { for (int i = 0; i < descriptor_->method_count(); i++) { const MethodDescriptor* method = descriptor_->method(i); + WriteMethodDocComment(printer, method); GenerateMethodSignature(printer, method, IS_ABSTRACT); printer->Print(";\n\n"); } diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.cc b/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.cc index 222285b..4815663 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.cc @@ -37,6 +37,7 @@ #include <string> #include <google/protobuf/compiler/java/java_string_field.h> +#include <google/protobuf/compiler/java/java_doc_comment.h> #include <google/protobuf/stubs/common.h> #include <google/protobuf/compiler/java/java_helpers.h> #include <google/protobuf/io/printer.h> @@ -85,6 +86,7 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, // For singular messages and builders, one bit is used for the hasField bit. (*variables)["get_has_field_bit_message"] = GenerateGetBit(messageBitIndex); + (*variables)["set_has_field_bit_message"] = GenerateSetBit(messageBitIndex); (*variables)["get_has_field_bit_builder"] = GenerateGetBit(builderBitIndex); (*variables)["set_has_field_bit_builder"] = GenerateSetBit(builderBitIndex); @@ -96,6 +98,13 @@ void SetPrimitiveVariables(const FieldDescriptor* descriptor, (*variables)["set_mutable_bit_builder"] = GenerateSetBit(builderBitIndex); (*variables)["clear_mutable_bit_builder"] = GenerateClearBit(builderBitIndex); + // For repeated fields, one bit is used for whether the array is immutable + // in the parsing constructor. + (*variables)["get_mutable_bit_parser"] = + GenerateGetBitMutableLocal(builderBitIndex); + (*variables)["set_mutable_bit_parser"] = + GenerateSetBitMutableLocal(builderBitIndex); + (*variables)["get_has_field_bit_from_local"] = GenerateGetBitFromLocal(builderBitIndex); (*variables)["set_has_field_bit_to_local"] = @@ -160,19 +169,29 @@ int StringFieldGenerator::GetNumBitsForBuilder() const { // UnmodifiableLazyStringList. void StringFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$boolean has$capitalized_name$();\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "$deprecation$boolean has$capitalized_name$();\n" "$deprecation$java.lang.String get$capitalized_name$();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes();\n"); } void StringFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private java.lang.Object $name$_;\n" + "private java.lang.Object $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_message$;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public java.lang.String get$capitalized_name$() {\n" " java.lang.Object ref = $name$_;\n" @@ -182,13 +201,16 @@ GenerateMembers(io::Printer* printer) const { " com.google.protobuf.ByteString bs = \n" " (com.google.protobuf.ByteString) ref;\n" " java.lang.String s = bs.toStringUtf8();\n" - " if (com.google.protobuf.Internal.isValidUtf8(bs)) {\n" + " if (bs.isValidUtf8()) {\n" " $name$_ = s;\n" " }\n" " return s;\n" " }\n" - "}\n" - "private com.google.protobuf.ByteString get$capitalized_name$Bytes() {\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$public com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes() {\n" " java.lang.Object ref = $name$_;\n" " if (ref instanceof java.lang.String) {\n" " com.google.protobuf.ByteString b = \n" @@ -205,11 +227,14 @@ GenerateMembers(io::Printer* printer) const { void StringFieldGenerator:: GenerateBuilderMembers(io::Printer* printer) const { printer->Print(variables_, - "private java.lang.Object $name$_ $default_init$;\n" + "private java.lang.Object $name$_ $default_init$;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public boolean has$capitalized_name$() {\n" " return $get_has_field_bit_builder$;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public java.lang.String get$capitalized_name$() {\n" " java.lang.Object ref = $name$_;\n" @@ -223,6 +248,23 @@ GenerateBuilderMembers(io::Printer* printer) const { " }\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$public com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes() {\n" + " java.lang.Object ref = $name$_;\n" + " if (ref instanceof String) {\n" + " com.google.protobuf.ByteString b = \n" + " com.google.protobuf.ByteString.copyFromUtf8(\n" + " (java.lang.String) ref);\n" + " $name$_ = b;\n" + " return b;\n" + " } else {\n" + " return (com.google.protobuf.ByteString) ref;\n" + " }\n" + "}\n"); + + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$(\n" " java.lang.String value) {\n" @@ -231,7 +273,9 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_ = value;\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $clear_has_field_bit_builder$;\n"); // The default value is not a simple literal so we want to avoid executing @@ -243,11 +287,15 @@ GenerateBuilderMembers(io::Printer* printer) const { " return this;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "void set$capitalized_name$(com.google.protobuf.ByteString value) {\n" + "$deprecation$public Builder set$capitalized_name$Bytes(\n" + " com.google.protobuf.ByteString value) {\n" + "$null_check$" " $set_has_field_bit_builder$;\n" " $name$_ = value;\n" " $on_changed$\n" + " return this;\n" "}\n"); } @@ -270,9 +318,13 @@ GenerateBuilderClearCode(io::Printer* printer) const { void StringFieldGenerator:: GenerateMergingCode(io::Printer* printer) const { + // Allow a slight breach of abstraction here in order to avoid forcing + // all string fields to Strings when copying fields from a Message. printer->Print(variables_, "if (other.has$capitalized_name$()) {\n" - " set$capitalized_name$(other.get$capitalized_name$());\n" + " $set_has_field_bit_builder$;\n" + " $name$_ = other.$name$_;\n" + " $on_changed$\n" "}\n"); } @@ -288,11 +340,16 @@ GenerateBuildingCode(io::Printer* printer) const { void StringFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "$set_has_field_bit_builder$;\n" + "$set_has_field_bit_message$;\n" "$name$_ = input.readBytes();\n"); } void StringFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + // noop for strings. +} + +void StringFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { printer->Print(variables_, "if ($get_has_field_bit_message$) {\n" @@ -353,28 +410,49 @@ int RepeatedStringFieldGenerator::GetNumBitsForBuilder() const { void RepeatedStringFieldGenerator:: GenerateInterfaceMembers(io::Printer* printer) const { + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$java.util.List<java.lang.String>\n" - " get$capitalized_name$List();\n" - "$deprecation$int get$capitalized_name$Count();\n" + "get$capitalized_name$List();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$int get$capitalized_name$Count();\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$java.lang.String get$capitalized_name$(int index);\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes(int index);\n"); } void RepeatedStringFieldGenerator:: GenerateMembers(io::Printer* printer) const { printer->Print(variables_, - "private com.google.protobuf.LazyStringList $name$_;\n" + "private com.google.protobuf.LazyStringList $name$_;\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.util.List<java.lang.String>\n" " get$capitalized_name$List() {\n" " return $name$_;\n" // note: unmodifiable list - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.lang.String get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$public com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes(int index) {\n" + " return $name$_.getByteString(index);\n" + "}\n"); if (descriptor_->options().packed() && HasGeneratedMethods(descriptor_->containing_type())) { @@ -409,17 +487,30 @@ GenerateBuilderMembers(io::Printer* printer) const { // could hold on to the returned list and modify it after the message // has been built, thus mutating the message which is supposed to be // immutable. + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, "$deprecation$public java.util.List<java.lang.String>\n" " get$capitalized_name$List() {\n" " return java.util.Collections.unmodifiableList($name$_);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public int get$capitalized_name$Count() {\n" " return $name$_.size();\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public java.lang.String get$capitalized_name$(int index) {\n" " return $name$_.get(index);\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, + "$deprecation$public com.google.protobuf.ByteString\n" + " get$capitalized_name$Bytes(int index) {\n" + " return $name$_.getByteString(index);\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder set$capitalized_name$(\n" " int index, java.lang.String value) {\n" "$null_check$" @@ -427,7 +518,9 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_.set(index, value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder add$capitalized_name$(\n" " java.lang.String value) {\n" "$null_check$" @@ -435,14 +528,18 @@ GenerateBuilderMembers(io::Printer* printer) const { " $name$_.add(value);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder addAll$capitalized_name$(\n" " java.lang.Iterable<java.lang.String> values) {\n" " ensure$capitalized_name$IsMutable();\n" " super.addAll(values, $name$_);\n" " $on_changed$\n" " return this;\n" - "}\n" + "}\n"); + WriteFieldDocComment(printer, descriptor_); + printer->Print(variables_, "$deprecation$public Builder clear$capitalized_name$() {\n" " $name$_ = $empty_list$;\n" " $clear_mutable_bit_builder$;\n" @@ -450,11 +547,15 @@ GenerateBuilderMembers(io::Printer* printer) const { " return this;\n" "}\n"); + WriteFieldDocComment(printer, descriptor_); printer->Print(variables_, - "void add$capitalized_name$(com.google.protobuf.ByteString value) {\n" + "$deprecation$public Builder add$capitalized_name$Bytes(\n" + " com.google.protobuf.ByteString value) {\n" + "$null_check$" " ensure$capitalized_name$IsMutable();\n" " $name$_.add(value);\n" " $on_changed$\n" + " return this;\n" "}\n"); } @@ -512,7 +613,10 @@ GenerateBuildingCode(io::Printer* printer) const { void RepeatedStringFieldGenerator:: GenerateParsingCode(io::Printer* printer) const { printer->Print(variables_, - "ensure$capitalized_name$IsMutable();\n" + "if (!$get_mutable_bit_parser$) {\n" + " $name$_ = new com.google.protobuf.LazyStringArrayList();\n" + " $set_mutable_bit_parser$;\n" + "}\n" "$name$_.add(input.readBytes());\n"); } @@ -521,13 +625,25 @@ GenerateParsingCodeFromPacked(io::Printer* printer) const { printer->Print(variables_, "int length = input.readRawVarint32();\n" "int limit = input.pushLimit(length);\n" + "if (!$get_mutable_bit_parser$ && input.getBytesUntilLimit() > 0) {\n" + " $name$_ = new com.google.protobuf.LazyStringArrayList();\n" + " $set_mutable_bit_parser$;\n" + "}\n" "while (input.getBytesUntilLimit() > 0) {\n" - " add$capitalized_name$(input.read$capitalized_type$());\n" + " $name$.add(input.read$capitalized_type$());\n" "}\n" "input.popLimit(limit);\n"); } void RepeatedStringFieldGenerator:: +GenerateParsingDoneCode(io::Printer* printer) const { + printer->Print(variables_, + "if ($get_mutable_bit_parser$) {\n" + " $name$_ = new com.google.protobuf.UnmodifiableLazyStringList($name$_);\n" + "}\n"); +} + +void RepeatedStringFieldGenerator:: GenerateSerializationCode(io::Printer* printer) const { if (descriptor_->options().packed()) { printer->Print(variables_, diff --git a/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.h b/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.h index 8cb4146..4f7532f 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.h +++ b/third_party/protobuf/src/google/protobuf/compiler/java/java_string_field.h @@ -62,6 +62,7 @@ class StringFieldGenerator : public FieldGenerator { void GenerateMergingCode(io::Printer* printer) const; void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; @@ -96,6 +97,7 @@ class RepeatedStringFieldGenerator : public FieldGenerator { void GenerateBuildingCode(io::Printer* printer) const; void GenerateParsingCode(io::Printer* printer) const; void GenerateParsingCodeFromPacked(io::Printer* printer) const; + void GenerateParsingDoneCode(io::Printer* printer) const; void GenerateSerializationCode(io::Printer* printer) const; void GenerateSerializedSizeCode(io::Printer* printer) const; void GenerateFieldBuilderInitializationCode(io::Printer* printer) const; diff --git a/third_party/protobuf/src/google/protobuf/compiler/main.cc b/third_party/protobuf/src/google/protobuf/compiler/main.cc index d9b0c3f..1afc5d6 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/main.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/main.cc @@ -43,7 +43,7 @@ int main(int argc, char* argv[]) { // Proto2 C++ google::protobuf::compiler::cpp::CppGenerator cpp_generator; - cli.RegisterGenerator("--cpp_out", &cpp_generator, + cli.RegisterGenerator("--cpp_out", "--cpp_opt", &cpp_generator, "Generate C++ header and source."); // Proto2 Java diff --git a/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.cc b/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.cc index 5b76af2..0e35ed1 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.cc @@ -33,13 +33,14 @@ #include <google/protobuf/compiler/mock_code_generator.h> #include <google/protobuf/testing/file.h> +#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/descriptor.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -132,6 +133,15 @@ bool MockCodeGenerator::Generate( } else if (command == "Abort") { cerr << "Saw message type MockCodeGenerator_Abort." << endl; abort(); + } else if (command == "HasSourceCodeInfo") { + FileDescriptorProto file_descriptor_proto; + file->CopySourceCodeInfoTo(&file_descriptor_proto); + bool has_source_code_info = + file_descriptor_proto.has_source_code_info() && + file_descriptor_proto.source_code_info().location_size() > 0; + cerr << "Saw message type MockCodeGenerator_HasSourceCodeInfo: " + << has_source_code_info << "." << endl; + abort(); } else { GOOGLE_LOG(FATAL) << "Unknown MockCodeGenerator command: " << command; } diff --git a/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.h b/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.h index 5c7942b..506fd20 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.h +++ b/third_party/protobuf/src/google/protobuf/compiler/mock_code_generator.h @@ -59,6 +59,10 @@ namespace compiler { // MockCodeGenerator_Exit." to stderr and then calls exit(123). // MockCodeGenerator_Abort: Generate() prints "Saw message type // MockCodeGenerator_Abort." to stderr and then calls abort(). +// MockCodeGenerator_HasSourceCodeInfo: Causes Generate() to abort after +// printing "Saw message type MockCodeGenerator_HasSourceCodeInfo: FOO." to +// stderr, where FOO is "1" if the supplied FileDescriptorProto has source +// code info, and "0" otherwise. class MockCodeGenerator : public CodeGenerator { public: MockCodeGenerator(const string& name); diff --git a/third_party/protobuf/src/google/protobuf/compiler/parser.cc b/third_party/protobuf/src/google/protobuf/compiler/parser.cc index 34317b1..23aa01c 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/parser.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/parser.cc @@ -174,6 +174,20 @@ bool Parser::ConsumeInteger(int* output, const char* error) { } } +bool Parser::ConsumeSignedInteger(int* output, const char* error) { + bool is_negative = false; + uint64 max_value = kint32max; + if (TryConsume("-")) { + is_negative = true; + max_value += 1; + } + uint64 value = 0; + DO(ConsumeInteger64(max_value, &value, error)); + if (is_negative) value *= -1; + *output = value; + return true; +} + bool Parser::ConsumeInteger64(uint64 max_value, uint64* output, const char* error) { if (LookingAtType(io::Tokenizer::TYPE_INTEGER)) { @@ -237,6 +251,35 @@ bool Parser::ConsumeString(string* output, const char* error) { } } +bool Parser::TryConsumeEndOfDeclaration(const char* text, + const LocationRecorder* location) { + if (LookingAt(text)) { + string leading, trailing; + input_->NextWithComments(&trailing, NULL, &leading); + + // Save the leading comments for next time, and recall the leading comments + // from last time. + leading.swap(upcoming_doc_comments_); + + if (location != NULL) { + location->AttachComments(&leading, &trailing); + } + return true; + } else { + return false; + } +} + +bool Parser::ConsumeEndOfDeclaration(const char* text, + const LocationRecorder* location) { + if (TryConsumeEndOfDeclaration(text, location)) { + return true; + } else { + AddError("Expected \"" + string(text) + "\"."); + return false; + } +} + // ------------------------------------------------------------------- void Parser::AddError(int line, int column, const string& error) { @@ -315,6 +358,19 @@ void Parser::LocationRecorder::RecordLegacyLocation(const Message* descriptor, } } +void Parser::LocationRecorder::AttachComments( + string* leading, string* trailing) const { + GOOGLE_CHECK(!location_->has_leading_comments()); + GOOGLE_CHECK(!location_->has_trailing_comments()); + + if (!leading->empty()) { + location_->mutable_leading_comments()->swap(*leading); + } + if (!trailing->empty()) { + location_->mutable_trailing_comments()->swap(*trailing); + } +} + // ------------------------------------------------------------------- void Parser::SkipStatement() { @@ -322,7 +378,7 @@ void Parser::SkipStatement() { if (AtEnd()) { return; } else if (LookingAtType(io::Tokenizer::TYPE_SYMBOL)) { - if (TryConsume(";")) { + if (TryConsumeEndOfDeclaration(";", NULL)) { return; } else if (TryConsume("{")) { SkipRestOfBlock(); @@ -340,7 +396,7 @@ void Parser::SkipRestOfBlock() { if (AtEnd()) { return; } else if (LookingAtType(io::Tokenizer::TYPE_SYMBOL)) { - if (TryConsume("}")) { + if (TryConsumeEndOfDeclaration("}", NULL)) { return; } else if (TryConsume("{")) { SkipRestOfBlock(); @@ -366,7 +422,7 @@ bool Parser::Parse(io::Tokenizer* input, FileDescriptorProto* file) { if (LookingAtType(io::Tokenizer::TYPE_START)) { // Advance to first token. - input_->Next(); + input_->NextWithComments(NULL, NULL, &upcoming_doc_comments_); } { @@ -393,7 +449,7 @@ bool Parser::Parse(io::Tokenizer* input, FileDescriptorProto* file) { if (LookingAt("}")) { AddError("Unmatched \"}\"."); - input_->Next(); + input_->NextWithComments(NULL, NULL, &upcoming_doc_comments_); } } } @@ -411,7 +467,7 @@ bool Parser::ParseSyntaxIdentifier() { io::Tokenizer::Token syntax_token = input_->current(); string syntax; DO(ConsumeString(&syntax, "Expected syntax identifier.")); - DO(Consume(";")); + DO(ConsumeEndOfDeclaration(";", NULL)); syntax_identifier_ = syntax; @@ -427,7 +483,7 @@ bool Parser::ParseSyntaxIdentifier() { bool Parser::ParseTopLevelStatement(FileDescriptorProto* file, const LocationRecorder& root_location) { - if (TryConsume(";")) { + if (TryConsumeEndOfDeclaration(";", NULL)) { // empty statement; ignore return true; } else if (LookingAt("message")) { @@ -451,14 +507,16 @@ bool Parser::ParseTopLevelStatement(FileDescriptorProto* file, FileDescriptorProto::kMessageTypeFieldNumber, location); } else if (LookingAt("import")) { - int index = file->dependency_size(); - return ParseImport(file->add_dependency(), root_location, index); + return ParseImport(file->mutable_dependency(), + file->mutable_public_dependency(), + file->mutable_weak_dependency(), + root_location); } else if (LookingAt("package")) { return ParsePackage(file, root_location); } else if (LookingAt("option")) { LocationRecorder location(root_location, FileDescriptorProto::kOptionsFieldNumber); - return ParseOption(file->mutable_options(), location); + return ParseOption(file->mutable_options(), location, OPTION_STATEMENT); } else { AddError("Expected top-level statement (e.g. \"message\")."); return false; @@ -482,11 +540,45 @@ bool Parser::ParseMessageDefinition(DescriptorProto* message, return true; } +namespace { + +const int kMaxExtensionRangeSentinel = -1; + +bool IsMessageSetWireFormatMessage(const DescriptorProto& message) { + const MessageOptions& options = message.options(); + for (int i = 0; i < options.uninterpreted_option_size(); ++i) { + const UninterpretedOption& uninterpreted = options.uninterpreted_option(i); + if (uninterpreted.name_size() == 1 && + uninterpreted.name(0).name_part() == "message_set_wire_format" && + uninterpreted.identifier_value() == "true") { + return true; + } + } + return false; +} + +// Modifies any extension ranges that specified 'max' as the end of the +// extension range, and sets them to the type-specific maximum. The actual max +// tag number can only be determined after all options have been parsed. +void AdjustExtensionRangesWithMaxEndNumber(DescriptorProto* message) { + const bool is_message_set = IsMessageSetWireFormatMessage(*message); + const int max_extension_number = is_message_set ? + kint32max : + FieldDescriptor::kMaxNumber + 1; + for (int i = 0; i < message->extension_range_size(); ++i) { + if (message->extension_range(i).end() == kMaxExtensionRangeSentinel) { + message->mutable_extension_range(i)->set_end(max_extension_number); + } + } +} + +} // namespace + bool Parser::ParseMessageBlock(DescriptorProto* message, const LocationRecorder& message_location) { - DO(Consume("{")); + DO(ConsumeEndOfDeclaration("{", &message_location)); - while (!TryConsume("}")) { + while (!TryConsumeEndOfDeclaration("}", NULL)) { if (AtEnd()) { AddError("Reached end of input in message definition (missing '}')."); return false; @@ -499,12 +591,15 @@ bool Parser::ParseMessageBlock(DescriptorProto* message, } } + if (message->extension_range_size() > 0) { + AdjustExtensionRangesWithMaxEndNumber(message); + } return true; } bool Parser::ParseMessageStatement(DescriptorProto* message, const LocationRecorder& message_location) { - if (TryConsume(";")) { + if (TryConsumeEndOfDeclaration(";", NULL)) { // empty statement; ignore return true; } else if (LookingAt("message")) { @@ -532,7 +627,7 @@ bool Parser::ParseMessageStatement(DescriptorProto* message, } else if (LookingAt("option")) { LocationRecorder location(message_location, DescriptorProto::kOptionsFieldNumber); - return ParseOption(message->mutable_options(), location); + return ParseOption(message->mutable_options(), location, OPTION_STATEMENT); } else { LocationRecorder location(message_location, DescriptorProto::kFieldFieldNumber, @@ -647,7 +742,7 @@ bool Parser::ParseMessageField(FieldDescriptorProto* field, return false; } } else { - DO(Consume(";")); + DO(ConsumeEndOfDeclaration(";", &field_location)); } return true; @@ -669,7 +764,7 @@ bool Parser::ParseFieldOptions(FieldDescriptorProto* field, // the default value is not actually an option. DO(ParseDefaultAssignment(field, field_location)); } else { - DO(ParseOptionAssignment(field->mutable_options(), location)); + DO(ParseOption(field->mutable_options(), location, OPTION_ASSIGNMENT)); } } while (TryConsume(",")); @@ -835,6 +930,8 @@ bool Parser::ParseOptionNamePart(UninterpretedOption* uninterpreted_option, bool Parser::ParseUninterpretedBlock(string* value) { // Note that enclosing braces are not added to *value. + // We do NOT use ConsumeEndOfStatement for this brace because it's delimiting + // an expression, not a block of statements. DO(Consume("{")); int brace_depth = 1; while (!AtEnd()) { @@ -858,8 +955,9 @@ bool Parser::ParseUninterpretedBlock(string* value) { // We don't interpret the option here. Instead we store it in an // UninterpretedOption, to be interpreted later. -bool Parser::ParseOptionAssignment(Message* options, - const LocationRecorder& options_location) { +bool Parser::ParseOption(Message* options, + const LocationRecorder& options_location, + OptionStyle style) { // Create an entry in the uninterpreted_option field. const FieldDescriptor* uninterpreted_option_field = options->GetDescriptor()-> FindFieldByName("uninterpreted_option"); @@ -872,6 +970,10 @@ bool Parser::ParseOptionAssignment(Message* options, options_location, uninterpreted_option_field->number(), reflection->FieldSize(*options, uninterpreted_option_field)); + if (style == OPTION_STATEMENT) { + DO(Consume("option")); + } + UninterpretedOption* uninterpreted_option = down_cast<UninterpretedOption*>( options->GetReflection()->AddMessage(options, uninterpreted_option_field)); @@ -899,82 +1001,91 @@ bool Parser::ParseOptionAssignment(Message* options, DO(Consume("=")); - LocationRecorder value_location(location); - value_location.RecordLegacyLocation( - uninterpreted_option, DescriptorPool::ErrorCollector::OPTION_VALUE); - - // All values are a single token, except for negative numbers, which consist - // of a single '-' symbol, followed by a positive number. - bool is_negative = TryConsume("-"); + { + LocationRecorder value_location(location); + value_location.RecordLegacyLocation( + uninterpreted_option, DescriptorPool::ErrorCollector::OPTION_VALUE); - switch (input_->current().type) { - case io::Tokenizer::TYPE_START: - GOOGLE_LOG(FATAL) << "Trying to read value before any tokens have been read."; - return false; + // All values are a single token, except for negative numbers, which consist + // of a single '-' symbol, followed by a positive number. + bool is_negative = TryConsume("-"); - case io::Tokenizer::TYPE_END: - AddError("Unexpected end of stream while parsing option value."); - return false; + switch (input_->current().type) { + case io::Tokenizer::TYPE_START: + GOOGLE_LOG(FATAL) << "Trying to read value before any tokens have been read."; + return false; - case io::Tokenizer::TYPE_IDENTIFIER: { - value_location.AddPath(UninterpretedOption::kIdentifierValueFieldNumber); - if (is_negative) { - AddError("Invalid '-' symbol before identifier."); + case io::Tokenizer::TYPE_END: + AddError("Unexpected end of stream while parsing option value."); return false; - } - string value; - DO(ConsumeIdentifier(&value, "Expected identifier.")); - uninterpreted_option->set_identifier_value(value); - break; - } - case io::Tokenizer::TYPE_INTEGER: { - uint64 value; - uint64 max_value = - is_negative ? static_cast<uint64>(kint64max) + 1 : kuint64max; - DO(ConsumeInteger64(max_value, &value, "Expected integer.")); - if (is_negative) { + case io::Tokenizer::TYPE_IDENTIFIER: { value_location.AddPath( - UninterpretedOption::kNegativeIntValueFieldNumber); - uninterpreted_option->set_negative_int_value(-static_cast<int64>(value)); - } else { - value_location.AddPath( - UninterpretedOption::kPositiveIntValueFieldNumber); - uninterpreted_option->set_positive_int_value(value); + UninterpretedOption::kIdentifierValueFieldNumber); + if (is_negative) { + AddError("Invalid '-' symbol before identifier."); + return false; + } + string value; + DO(ConsumeIdentifier(&value, "Expected identifier.")); + uninterpreted_option->set_identifier_value(value); + break; } - break; - } - case io::Tokenizer::TYPE_FLOAT: { - value_location.AddPath(UninterpretedOption::kDoubleValueFieldNumber); - double value; - DO(ConsumeNumber(&value, "Expected number.")); - uninterpreted_option->set_double_value(is_negative ? -value : value); - break; - } + case io::Tokenizer::TYPE_INTEGER: { + uint64 value; + uint64 max_value = + is_negative ? static_cast<uint64>(kint64max) + 1 : kuint64max; + DO(ConsumeInteger64(max_value, &value, "Expected integer.")); + if (is_negative) { + value_location.AddPath( + UninterpretedOption::kNegativeIntValueFieldNumber); + uninterpreted_option->set_negative_int_value( + -static_cast<int64>(value)); + } else { + value_location.AddPath( + UninterpretedOption::kPositiveIntValueFieldNumber); + uninterpreted_option->set_positive_int_value(value); + } + break; + } - case io::Tokenizer::TYPE_STRING: { - value_location.AddPath(UninterpretedOption::kStringValueFieldNumber); - if (is_negative) { - AddError("Invalid '-' symbol before string."); - return false; + case io::Tokenizer::TYPE_FLOAT: { + value_location.AddPath(UninterpretedOption::kDoubleValueFieldNumber); + double value; + DO(ConsumeNumber(&value, "Expected number.")); + uninterpreted_option->set_double_value(is_negative ? -value : value); + break; } - string value; - DO(ConsumeString(&value, "Expected string.")); - uninterpreted_option->set_string_value(value); - break; - } - case io::Tokenizer::TYPE_SYMBOL: - if (LookingAt("{")) { - value_location.AddPath(UninterpretedOption::kAggregateValueFieldNumber); - DO(ParseUninterpretedBlock( - uninterpreted_option->mutable_aggregate_value())); - } else { - AddError("Expected option value."); - return false; + case io::Tokenizer::TYPE_STRING: { + value_location.AddPath(UninterpretedOption::kStringValueFieldNumber); + if (is_negative) { + AddError("Invalid '-' symbol before string."); + return false; + } + string value; + DO(ConsumeString(&value, "Expected string.")); + uninterpreted_option->set_string_value(value); + break; } - break; + + case io::Tokenizer::TYPE_SYMBOL: + if (LookingAt("{")) { + value_location.AddPath( + UninterpretedOption::kAggregateValueFieldNumber); + DO(ParseUninterpretedBlock( + uninterpreted_option->mutable_aggregate_value())); + } else { + AddError("Expected option value."); + return false; + } + break; + } + } + + if (style == OPTION_STATEMENT) { + DO(ConsumeEndOfDeclaration(";", &location)); } return true; @@ -1008,7 +1119,10 @@ bool Parser::ParseExtensions(DescriptorProto* message, LocationRecorder end_location( location, DescriptorProto::ExtensionRange::kEndFieldNumber); if (TryConsume("max")) { - end = FieldDescriptor::kMaxNumber; + // Set to the sentinel value - 1 since we increment the value below. + // The actual value of the end of the range should be set with + // AdjustExtensionRangesWithMaxEndNumber. + end = kMaxExtensionRangeSentinel - 1; } else { DO(ConsumeInteger(&end, "Expected integer.")); } @@ -1028,7 +1142,7 @@ bool Parser::ParseExtensions(DescriptorProto* message, range->set_end(end); } while (TryConsume(",")); - DO(Consume(";")); + DO(ConsumeEndOfDeclaration(";", &extensions_location)); return true; } @@ -1046,7 +1160,7 @@ bool Parser::ParseExtend(RepeatedPtrField<FieldDescriptorProto>* extensions, io::Tokenizer::Token extendee_end = input_->previous(); // Parse the block. - DO(Consume("{")); + DO(ConsumeEndOfDeclaration("{", &extend_location)); bool is_first = true; @@ -1083,7 +1197,7 @@ bool Parser::ParseExtend(RepeatedPtrField<FieldDescriptorProto>* extensions, // other statements. SkipStatement(); } - } while(!TryConsume("}")); + } while (!TryConsumeEndOfDeclaration("}", NULL)); return true; } @@ -1109,9 +1223,9 @@ bool Parser::ParseEnumDefinition(EnumDescriptorProto* enum_type, bool Parser::ParseEnumBlock(EnumDescriptorProto* enum_type, const LocationRecorder& enum_location) { - DO(Consume("{")); + DO(ConsumeEndOfDeclaration("{", &enum_location)); - while (!TryConsume("}")) { + while (!TryConsumeEndOfDeclaration("}", NULL)) { if (AtEnd()) { AddError("Reached end of input in enum definition (missing '}')."); return false; @@ -1129,13 +1243,14 @@ bool Parser::ParseEnumBlock(EnumDescriptorProto* enum_type, bool Parser::ParseEnumStatement(EnumDescriptorProto* enum_type, const LocationRecorder& enum_location) { - if (TryConsume(";")) { + if (TryConsumeEndOfDeclaration(";", NULL)) { // empty statement; ignore return true; } else if (LookingAt("option")) { LocationRecorder location(enum_location, EnumDescriptorProto::kOptionsFieldNumber); - return ParseOption(enum_type->mutable_options(), location); + return ParseOption(enum_type->mutable_options(), location, + OPTION_STATEMENT); } else { LocationRecorder location(enum_location, EnumDescriptorProto::kValueFieldNumber, enum_type->value_size()); @@ -1164,16 +1279,14 @@ bool Parser::ParseEnumConstant(EnumValueDescriptorProto* enum_value, location.RecordLegacyLocation( enum_value, DescriptorPool::ErrorCollector::NUMBER); - bool is_negative = TryConsume("-"); int number; - DO(ConsumeInteger(&number, "Expected integer.")); - if (is_negative) number *= -1; + DO(ConsumeSignedInteger(&number, "Expected integer.")); enum_value->set_number(number); } DO(ParseEnumConstantOptions(enum_value, enum_value_location)); - DO(Consume(";")); + DO(ConsumeEndOfDeclaration(";", &enum_value_location)); return true; } @@ -1189,7 +1302,7 @@ bool Parser::ParseEnumConstantOptions( DO(Consume("[")); do { - DO(ParseOptionAssignment(value->mutable_options(), location)); + DO(ParseOption(value->mutable_options(), location, OPTION_ASSIGNMENT)); } while (TryConsume(",")); DO(Consume("]")); @@ -1217,9 +1330,9 @@ bool Parser::ParseServiceDefinition(ServiceDescriptorProto* service, bool Parser::ParseServiceBlock(ServiceDescriptorProto* service, const LocationRecorder& service_location) { - DO(Consume("{")); + DO(ConsumeEndOfDeclaration("{", &service_location)); - while (!TryConsume("}")) { + while (!TryConsumeEndOfDeclaration("}", NULL)) { if (AtEnd()) { AddError("Reached end of input in service definition (missing '}')."); return false; @@ -1237,13 +1350,13 @@ bool Parser::ParseServiceBlock(ServiceDescriptorProto* service, bool Parser::ParseServiceStatement(ServiceDescriptorProto* service, const LocationRecorder& service_location) { - if (TryConsume(";")) { + if (TryConsumeEndOfDeclaration(";", NULL)) { // empty statement; ignore return true; } else if (LookingAt("option")) { LocationRecorder location( service_location, ServiceDescriptorProto::kOptionsFieldNumber); - return ParseOption(service->mutable_options(), location); + return ParseOption(service->mutable_options(), location, OPTION_STATEMENT); } else { LocationRecorder location(service_location, ServiceDescriptorProto::kMethodFieldNumber, service->method_size()); @@ -1286,28 +1399,41 @@ bool Parser::ParseServiceMethod(MethodDescriptorProto* method, } DO(Consume(")")); - if (TryConsume("{")) { + if (LookingAt("{")) { // Options! - while (!TryConsume("}")) { - if (AtEnd()) { - AddError("Reached end of input in method options (missing '}')."); - return false; - } + DO(ParseOptions(method_location, + MethodDescriptorProto::kOptionsFieldNumber, + method->mutable_options())); + } else { + DO(ConsumeEndOfDeclaration(";", &method_location)); + } - if (TryConsume(";")) { - // empty statement; ignore - } else { - LocationRecorder location(method_location, - MethodDescriptorProto::kOptionsFieldNumber); - if (!ParseOption(method->mutable_options(), location)) { - // This statement failed to parse. Skip it, but keep looping to - // parse other statements. - SkipStatement(); - } + return true; +} + + +bool Parser::ParseOptions(const LocationRecorder& parent_location, + const int optionsFieldNumber, + Message* mutable_options) { + // Options! + ConsumeEndOfDeclaration("{", &parent_location); + while (!TryConsumeEndOfDeclaration("}", NULL)) { + if (AtEnd()) { + AddError("Reached end of input in method options (missing '}')."); + return false; + } + + if (TryConsumeEndOfDeclaration(";", NULL)) { + // empty statement; ignore + } else { + LocationRecorder location(parent_location, + optionsFieldNumber); + if (!ParseOption(mutable_options, location, OPTION_STATEMENT)) { + // This statement failed to parse. Skip it, but keep looping to + // parse other statements. + SkipStatement(); } } - } else { - DO(Consume(";")); } return true; @@ -1406,32 +1532,44 @@ bool Parser::ParsePackage(FileDescriptorProto* file, if (!TryConsume(".")) break; file->mutable_package()->append("."); } + + location.EndAt(input_->previous()); + + DO(ConsumeEndOfDeclaration(";", &location)); } - DO(Consume(";")); return true; } -bool Parser::ParseImport(string* import_filename, - const LocationRecorder& root_location, - int index) { +bool Parser::ParseImport(RepeatedPtrField<string>* dependency, + RepeatedField<int32>* public_dependency, + RepeatedField<int32>* weak_dependency, + const LocationRecorder& root_location) { DO(Consume("import")); + if (LookingAt("public")) { + LocationRecorder location( + root_location, FileDescriptorProto::kPublicDependencyFieldNumber, + public_dependency->size()); + DO(Consume("public")); + *public_dependency->Add() = dependency->size(); + } else if (LookingAt("weak")) { + LocationRecorder location( + root_location, FileDescriptorProto::kWeakDependencyFieldNumber, + weak_dependency->size()); + DO(Consume("weak")); + *weak_dependency->Add() = dependency->size(); + } { LocationRecorder location(root_location, FileDescriptorProto::kDependencyFieldNumber, - index); - DO(ConsumeString(import_filename, + dependency->size()); + DO(ConsumeString(dependency->Add(), "Expected a string naming the file to import.")); - } - DO(Consume(";")); - return true; -} -bool Parser::ParseOption(Message* options, - const LocationRecorder& options_location) { - DO(Consume("option")); - DO(ParseOptionAssignment(options, options_location)); - DO(Consume(";")); + location.EndAt(input_->previous()); + + DO(ConsumeEndOfDeclaration(";", &location)); + } return true; } diff --git a/third_party/protobuf/src/google/protobuf/compiler/parser.h b/third_party/protobuf/src/google/protobuf/compiler/parser.h index 4cc90a29..cfd3649 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/parser.h +++ b/third_party/protobuf/src/google/protobuf/compiler/parser.h @@ -116,6 +116,8 @@ class LIBPROTOBUF_EXPORT Parser { } private: + class LocationRecorder; + // ================================================================= // Error recovery helpers @@ -164,6 +166,8 @@ class LIBPROTOBUF_EXPORT Parser { bool ConsumeIdentifier(string* output, const char* error); // Consume an integer and store its value in "output". bool ConsumeInteger(int* output, const char* error); + // Consume a signed integer and store its value in "output". + bool ConsumeSignedInteger(int* output, const char* error); // Consume a 64-bit integer and store its value in "output". If the value // is greater than max_value, an error will be reported. bool ConsumeInteger64(uint64 max_value, uint64* output, const char* error); @@ -173,6 +177,20 @@ class LIBPROTOBUF_EXPORT Parser { // Consume a string literal and store its (unescaped) value in "output". bool ConsumeString(string* output, const char* error); + // Consume a token representing the end of the statement. Comments between + // this token and the next will be harvested for documentation. The given + // LocationRecorder should refer to the declaration that was just parsed; + // it will be populated with these comments. + // + // TODO(kenton): The LocationRecorder is const because historically locations + // have been passed around by const reference, for no particularly good + // reason. We should probably go through and change them all to mutable + // pointer to make this more intuitive. + bool TryConsumeEndOfDeclaration(const char* text, + const LocationRecorder* location); + bool ConsumeEndOfDeclaration(const char* text, + const LocationRecorder* location); + // ----------------------------------------------------------------- // Error logging helpers @@ -227,6 +245,14 @@ class LIBPROTOBUF_EXPORT Parser { void RecordLegacyLocation(const Message* descriptor, DescriptorPool::ErrorCollector::ErrorLocation location); + // Attaches leading and trailing comments to the location. The two strings + // will be swapped into place, so after this is called *leading and + // *trailing will be empty. + // + // TODO(kenton): See comment on TryConsumeEndOfDeclaration(), above, for + // why this is const. + void AttachComments(string* leading, string* trailing) const; + private: Parser* parser_; SourceCodeInfo::Location* location_; @@ -265,9 +291,10 @@ class LIBPROTOBUF_EXPORT Parser { const LocationRecorder& service_location); bool ParsePackage(FileDescriptorProto* file, const LocationRecorder& root_location); - bool ParseImport(string* import_filename, - const LocationRecorder& root_location, - int index); + bool ParseImport(RepeatedPtrField<string>* dependency, + RepeatedField<int32>* public_dependency, + RepeatedField<int32>* weak_dependency, + const LocationRecorder& root_location); bool ParseOption(Message* options, const LocationRecorder& options_location); @@ -329,6 +356,12 @@ class LIBPROTOBUF_EXPORT Parser { bool ParseServiceMethod(MethodDescriptorProto* method, const LocationRecorder& method_location); + + // Parse options of a single method or stream. + bool ParseOptions(const LocationRecorder& parent_location, + const int optionsFieldNumber, + Message* mutable_options); + // Parse "required", "optional", or "repeated" and fill in "label" // with the value. bool ParseLabel(FieldDescriptorProto::Label* label); @@ -351,11 +384,17 @@ class LIBPROTOBUF_EXPORT Parser { bool ParseDefaultAssignment(FieldDescriptorProto* field, const LocationRecorder& field_location); + enum OptionStyle { + OPTION_ASSIGNMENT, // just "name = value" + OPTION_STATEMENT // "option name = value;" + }; + // Parse a single option name/value pair, e.g. "ctype = CORD". The name // identifies a field of the given Message, and the value of that field // is set to the parsed value. - bool ParseOptionAssignment(Message* options, - const LocationRecorder& options_location); + bool ParseOption(Message* options, + const LocationRecorder& options_location, + OptionStyle style); // Parses a single part of a multipart option name. A multipart name consists // of names separated by dots. Each name is either an identifier or a series @@ -387,6 +426,10 @@ class LIBPROTOBUF_EXPORT Parser { bool stop_after_syntax_identifier_; string syntax_identifier_; + // Leading doc comments for the next declaration. These are not complete + // yet; use ConsumeEndOfDeclaration() to get the complete comments. + string upcoming_doc_comments_; + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(Parser); }; diff --git a/third_party/protobuf/src/google/protobuf/compiler/parser_unittest.cc b/third_party/protobuf/src/google/protobuf/compiler/parser_unittest.cc index 156c0dc..c61ac60 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/parser_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/parser_unittest.cc @@ -44,6 +44,7 @@ #include <google/protobuf/wire_format.h> #include <google/protobuf/text_format.h> #include <google/protobuf/unittest.pb.h> +#include <google/protobuf/unittest_custom_options.pb.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> #include <google/protobuf/stubs/map-util.h> @@ -509,6 +510,31 @@ TEST_F(ParseMessageTest, CompoundExtensionRange) { "}"); } +TEST_F(ParseMessageTest, LargerMaxForMessageSetWireFormatMessages) { + // Messages using the message_set_wire_format option can accept larger + // extension numbers, as the numbers are not encoded as int32 field values + // rather than tags. + ExpectParsesTo( + "message TestMessage {\n" + " extensions 4 to max;\n" + " option message_set_wire_format = true;\n" + "}\n", + + "message_type {" + " name: \"TestMessage\"" + " extension_range { start:4 end: 0x7fffffff }" + " options {\n" + " uninterpreted_option { \n" + " name {\n" + " name_part: \"message_set_wire_format\"\n" + " is_extension: false\n" + " }\n" + " identifier_value: \"true\"\n" + " }\n" + " }\n" + "}"); +} + TEST_F(ParseMessageTest, Extensions) { ExpectParsesTo( "extend Extendee1 { optional int32 foo = 12; }\n" @@ -571,6 +597,10 @@ TEST_F(ParseEnumTest, Values) { " FOO = 13;\n" " BAR = -10;\n" " BAZ = 500;\n" + " HEX_MAX = 0x7FFFFFFF;\n" + " HEX_MIN = -0x80000000;\n" + " INT_MAX = 2147483647;\n" + " INT_MIN = -2147483648;\n" "}\n", "enum_type {" @@ -578,6 +608,10 @@ TEST_F(ParseEnumTest, Values) { " value { name:\"FOO\" number:13 }" " value { name:\"BAR\" number:-10 }" " value { name:\"BAZ\" number:500 }" + " value { name:\"HEX_MAX\" number:2147483647 }" + " value { name:\"HEX_MIN\" number:-2147483648 }" + " value { name:\"INT_MAX\" number:2147483647 }" + " value { name:\"INT_MIN\" number:-2147483648 }" "}"); } @@ -631,7 +665,7 @@ TEST_F(ParseServiceTest, SimpleService) { "}"); } -TEST_F(ParseServiceTest, Methods) { +TEST_F(ParseServiceTest, MethodsAndStreams) { ExpectParsesTo( "service TestService {\n" " rpc Foo(In1) returns (Out1);\n" @@ -668,6 +702,20 @@ TEST_F(ParseMiscTest, ParseMultipleImports) { "dependency: \"baz.proto\""); } +TEST_F(ParseMiscTest, ParsePublicImports) { + ExpectParsesTo( + "import \"foo.proto\";\n" + "import public \"bar.proto\";\n" + "import \"baz.proto\";\n" + "import public \"qux.proto\";\n", + "dependency: \"foo.proto\"" + "dependency: \"bar.proto\"" + "dependency: \"baz.proto\"" + "dependency: \"qux.proto\"" + "public_dependency: 1 " + "public_dependency: 3 "); +} + TEST_F(ParseMiscTest, ParsePackage) { ExpectParsesTo( "package foo.bar.baz;\n", @@ -867,6 +915,20 @@ TEST_F(ParseErrorTest, DefaultValueTooLarge) { "6:36: Integer out of range.\n"); } +TEST_F(ParseErrorTest, EnumValueOutOfRange) { + ExpectHasErrors( + "enum TestEnum {\n" + " HEX_TOO_BIG = 0x80000000;\n" + " HEX_TOO_SMALL = -0x80000001;\n" + " INT_TOO_BIG = 2147483648;\n" + " INT_TOO_SMALL = -2147483649;\n" + "}\n", + "1:19: Integer out of range.\n" + "2:19: Integer out of range.\n" + "3:19: Integer out of range.\n" + "4:19: Integer out of range.\n"); +} + TEST_F(ParseErrorTest, DefaultValueMissing) { ExpectHasErrors( "message TestMessage {\n" @@ -976,6 +1038,7 @@ TEST_F(ParseErrorTest, ServiceMethodPrimitiveParams) { "1:26: Expected message type.\n"); } + TEST_F(ParseErrorTest, EofInMethodOptions) { ExpectHasErrors( "service TestService {\n" @@ -984,6 +1047,7 @@ TEST_F(ParseErrorTest, EofInMethodOptions) { "1:29: Reached end of input in service definition (missing '}').\n"); } + TEST_F(ParseErrorTest, PrimitiveMethodInput) { ExpectHasErrors( "service TestService {\n" @@ -992,6 +1056,7 @@ TEST_F(ParseErrorTest, PrimitiveMethodInput) { "1:10: Expected message type.\n"); } + TEST_F(ParseErrorTest, MethodOptionTypeError) { // This used to cause an infinite loop. ExpectHasErrors( @@ -1002,6 +1067,7 @@ TEST_F(ParseErrorTest, MethodOptionTypeError) { "2:45: Expected \"=\".\n"); } + // ------------------------------------------------------------------- // Import and package errors @@ -1157,6 +1223,7 @@ TEST_F(ParserValidationErrorTest, MethodNameError) { "3:6: \"Bar\" is already defined in \"Foo\".\n"); } + TEST_F(ParserValidationErrorTest, MethodInputTypeError) { ExpectHasValidationErrors( "message Baz {}\n" @@ -1166,6 +1233,7 @@ TEST_F(ParserValidationErrorTest, MethodInputTypeError) { "2:10: \"Qux\" is not defined.\n"); } + TEST_F(ParserValidationErrorTest, MethodOutputTypeError) { ExpectHasValidationErrors( "message Baz {}\n" @@ -1175,6 +1243,7 @@ TEST_F(ParserValidationErrorTest, MethodOutputTypeError) { "2:23: \"Qux\" is not defined.\n"); } + // =================================================================== // Test that the output from FileDescriptor::DebugString() (and all other // descriptor types) is parseable, and results in the same Descriptor @@ -1236,6 +1305,11 @@ TEST_F(ParseDecriptorDebugTest, TestAllDescriptorTypes) { // also need to give it the same name as the original. parsed.set_name("google/protobuf/unittest.proto"); // We need the imported dependency before we can build our parsed proto + const FileDescriptor* public_import = + protobuf_unittest_import::PublicImportMessage::descriptor()->file(); + FileDescriptorProto public_import_proto; + public_import->CopyTo(&public_import_proto); + ASSERT_TRUE(pool_.BuildFile(public_import_proto) != NULL); const FileDescriptor* import = protobuf_unittest_import::ImportMessage::descriptor()->file(); FileDescriptorProto import_proto; @@ -1258,6 +1332,45 @@ TEST_F(ParseDecriptorDebugTest, TestAllDescriptorTypes) { EXPECT_EQ(expected.DebugString(), parsed.DebugString()); } +TEST_F(ParseDecriptorDebugTest, TestCustomOptions) { + const FileDescriptor* original_file = + protobuf_unittest::AggregateMessage::descriptor()->file(); + FileDescriptorProto expected; + original_file->CopyTo(&expected); + + string debug_string = original_file->DebugString(); + + // Parse the debug string + SetupParser(debug_string.c_str()); + FileDescriptorProto parsed; + parser_->Parse(input_.get(), &parsed); + EXPECT_EQ(io::Tokenizer::TYPE_END, input_->current().type); + ASSERT_EQ("", error_collector_.text_); + + // We now have a FileDescriptorProto, but to compare with the expected we + // need to link to a FileDecriptor, then output back to a proto. We'll + // also need to give it the same name as the original. + parsed.set_name(original_file->name()); + + // unittest_custom_options.proto depends on descriptor.proto. + const FileDescriptor* import = FileDescriptorProto::descriptor()->file(); + FileDescriptorProto import_proto; + import->CopyTo(&import_proto); + ASSERT_TRUE(pool_.BuildFile(import_proto) != NULL); + const FileDescriptor* actual = pool_.BuildFile(parsed); + ASSERT_TRUE(actual != NULL); + parsed.Clear(); + actual->CopyTo(&parsed); + + // The messages might be in different orders, making them hard to compare. + // So, sort the messages in the descriptor protos (including nested messages, + // recursively). + SortMessages(&expected); + SortMessages(&parsed); + + EXPECT_EQ(expected.DebugString(), parsed.DebugString()); +} + // =================================================================== // SourceCodeInfo tests. @@ -1353,67 +1466,6 @@ bool FollowPath(const Message& root, } } -// Split some text on line breaks. The line breaks are retained in the output, -// so each line (except the last) ends with a '\n', and the lines can be -// concatenated to produce the original text. -// -// I couldn't find the proper string utility function for this. Our -// split-on-delimiter functions don't include the delimiter in the output. -void SplitLines(const string& text, vector<string>* lines) { - string::size_type pos = 0; - - while (pos != string::npos) { - string::size_type last_pos = pos; - pos = text.find_first_of('\n', pos); - if (pos != string::npos) ++pos; - lines->push_back(text.substr(last_pos, pos - last_pos)); - } -} - -// Look for the given tags in the given text and construct a span (as defined -// by SourceCodeInfo.Location.span) from them. E.g. for text like: -// /*a*/message /*b*/Foo/*c*/ {}/*d*/ -// There are four tags: "a", "b", "c", and "d". The constructed span starts -// immediately after the start tag's trailing '/' and ends immediately before -// the end tags leading '/'. -void MakeExpectedSpan(const vector<string>& lines, - const string& start_tag, const string& end_tag, - RepeatedField<int>* output) { - string start_comment = "/*" + start_tag + "*/"; - string end_comment = "/*" + end_tag + "*/"; - - int start_line = -1; - int start_column = -1; - for (int i = 0; i < lines.size(); i++) { - string::size_type pos = lines[i].find(start_comment); - if (pos != string::npos) { - start_line = i; - start_column = pos + start_comment.size(); - break; - } - } - ASSERT_NE(start_line, -1) - << "Tag \"" << start_comment << "\" not found in text."; - - int end_line = -1; - int end_column = -1; - for (int i = start_line; i < lines.size(); i++) { - string::size_type pos = lines[i].find(end_comment); - if (pos != string::npos) { - end_line = i; - end_column = pos; - break; - } - } - ASSERT_NE(end_line, -1) - << "Tag \"" << end_comment << "\" not found in text."; - - output->Add(start_line); - output->Add(start_column); - if (end_line != start_line) output->Add(end_line); - output->Add(end_column); -} - // Check if two spans are equal. bool CompareSpans(const RepeatedField<int>& span1, const RepeatedField<int>& span2) { @@ -1434,8 +1486,8 @@ class SourceInfoTest : public ParserTest { // Parse the given text as a .proto file and populate the spans_ map with // all the source location spans in its SourceCodeInfo table. bool Parse(const char* text) { - SetupParser(text); - SplitLines(text, &lines_); + ExtractMarkers(text); + SetupParser(text_without_markers_.c_str()); if (!parser_->Parse(input_.get(), &file_)) { return false; } @@ -1470,19 +1522,38 @@ class SourceInfoTest : public ParserTest { // part of the FileDescriptorProto. (If unclear, look at the actual tests; // it should quickly become obvious.) - bool HasSpan(const char* start_tag, const char* end_tag, + bool HasSpan(char start_marker, char end_marker, const Message& descriptor_proto) { - return HasSpan(start_tag, end_tag, descriptor_proto, NULL, -1); + return HasSpanWithComment( + start_marker, end_marker, descriptor_proto, NULL, -1, NULL, NULL); } - bool HasSpan(const char* start_tag, const char* end_tag, + bool HasSpanWithComment(char start_marker, char end_marker, + const Message& descriptor_proto, + const char* expected_leading_comments, + const char* expected_trailing_comments) { + return HasSpanWithComment( + start_marker, end_marker, descriptor_proto, NULL, -1, + expected_leading_comments, expected_trailing_comments); + } + + bool HasSpan(char start_marker, char end_marker, const Message& descriptor_proto, const string& field_name) { - return HasSpan(start_tag, end_tag, descriptor_proto, field_name, -1); + return HasSpan(start_marker, end_marker, descriptor_proto, field_name, -1); } - bool HasSpan(const char* start_tag, const char* end_tag, + bool HasSpan(char start_marker, char end_marker, const Message& descriptor_proto, const string& field_name, int index) { + return HasSpan(start_marker, end_marker, descriptor_proto, + field_name, index, NULL, NULL); + } + + bool HasSpan(char start_marker, char end_marker, + const Message& descriptor_proto, + const string& field_name, int index, + const char* expected_leading_comments, + const char* expected_trailing_comments) { const FieldDescriptor* field = descriptor_proto.GetDescriptor()->FindFieldByName(field_name); if (field == NULL) { @@ -1491,29 +1562,34 @@ class SourceInfoTest : public ParserTest { return false; } - return HasSpan(start_tag, end_tag, descriptor_proto, field, index); + return HasSpanWithComment( + start_marker, end_marker, descriptor_proto, field, index, + expected_leading_comments, expected_trailing_comments); } bool HasSpan(const Message& descriptor_proto) { - return HasSpan(NULL, NULL, descriptor_proto, NULL, -1); + return HasSpanWithComment( + '\0', '\0', descriptor_proto, NULL, -1, NULL, NULL); } bool HasSpan(const Message& descriptor_proto, const string& field_name) { - return HasSpan(NULL, NULL, descriptor_proto, field_name, -1); + return HasSpan('\0', '\0', descriptor_proto, field_name, -1); } bool HasSpan(const Message& descriptor_proto, const string& field_name, int index) { - return HasSpan(NULL, NULL, descriptor_proto, field_name, index); + return HasSpan('\0', '\0', descriptor_proto, field_name, index); } - bool HasSpan(const char* start_tag, const char* end_tag, - const Message& descriptor_proto, const FieldDescriptor* field, - int index) { + bool HasSpanWithComment(char start_marker, char end_marker, + const Message& descriptor_proto, + const FieldDescriptor* field, int index, + const char* expected_leading_comments, + const char* expected_trailing_comments) { pair<SpanMap::iterator, SpanMap::iterator> range = spans_.equal_range(SpanKey(descriptor_proto, field, index)); - if (start_tag == NULL) { + if (start_marker == '\0') { if (range.first == range.second) { return false; } else { @@ -1521,11 +1597,34 @@ class SourceInfoTest : public ParserTest { return true; } } else { + pair<int, int> start_pos = FindOrDie(markers_, start_marker); + pair<int, int> end_pos = FindOrDie(markers_, end_marker); + RepeatedField<int> expected_span; - MakeExpectedSpan(lines_, start_tag, end_tag, &expected_span); + expected_span.Add(start_pos.first); + expected_span.Add(start_pos.second); + if (end_pos.first != start_pos.first) { + expected_span.Add(end_pos.first); + } + expected_span.Add(end_pos.second); for (SpanMap::iterator iter = range.first; iter != range.second; ++iter) { if (CompareSpans(expected_span, iter->second->span())) { + if (expected_leading_comments == NULL) { + EXPECT_FALSE(iter->second->has_leading_comments()); + } else { + EXPECT_TRUE(iter->second->has_leading_comments()); + EXPECT_EQ(expected_leading_comments, + iter->second->leading_comments()); + } + if (expected_trailing_comments == NULL) { + EXPECT_FALSE(iter->second->has_trailing_comments()); + } else { + EXPECT_TRUE(iter->second->has_trailing_comments()); + EXPECT_EQ(expected_trailing_comments, + iter->second->trailing_comments()); + } + spans_.erase(iter); return true; } @@ -1542,10 +1641,11 @@ class SourceInfoTest : public ParserTest { int index; inline SpanKey() {} - inline SpanKey(const Message& descriptor_proto, - const FieldDescriptor* field, - int index) - : descriptor_proto(&descriptor_proto), field(field), index(index) {} + inline SpanKey(const Message& descriptor_proto_param, + const FieldDescriptor* field_param, + int index_param) + : descriptor_proto(&descriptor_proto_param), field(field_param), + index(index_param) {} inline bool operator<(const SpanKey& other) const { if (descriptor_proto < other.descriptor_proto) return true; @@ -1558,32 +1658,63 @@ class SourceInfoTest : public ParserTest { typedef multimap<SpanKey, const SourceCodeInfo::Location*> SpanMap; SpanMap spans_; - vector<string> lines_; + map<char, pair<int, int> > markers_; + string text_without_markers_; + + void ExtractMarkers(const char* text) { + markers_.clear(); + text_without_markers_.clear(); + int line = 0; + int column = 0; + while (*text != '\0') { + if (*text == '$') { + ++text; + GOOGLE_CHECK_NE('\0', *text); + if (*text == '$') { + text_without_markers_ += '$'; + ++column; + } else { + markers_[*text] = make_pair(line, column); + ++text; + GOOGLE_CHECK_EQ('$', *text); + } + } else if (*text == '\n') { + ++line; + column = 0; + text_without_markers_ += *text; + } else { + text_without_markers_ += *text; + ++column; + } + ++text; + } + } }; TEST_F(SourceInfoTest, BasicFileDecls) { EXPECT_TRUE(Parse( - "/*a*/syntax = \"proto2\";\n" - "package /*b*/foo.bar/*c*/;\n" - "import /*d*/\"baz.proto\"/*e*/;\n" - "import /*f*/\"qux.proto\"/*g*/;/*h*/\n" + "$a$syntax = \"proto2\";\n" + "package $b$foo.bar$c$;\n" + "import $d$\"baz.proto\"$e$;\n" + "import $f$\"qux.proto\"$g$;$h$\n" + "\n" "// comment ignored\n")); - EXPECT_TRUE(HasSpan("a", "h", file_)); - EXPECT_TRUE(HasSpan("b", "c", file_, "package")); - EXPECT_TRUE(HasSpan("d", "e", file_, "dependency", 0)); - EXPECT_TRUE(HasSpan("f", "g", file_, "dependency", 1)); + EXPECT_TRUE(HasSpan('a', 'h', file_)); + EXPECT_TRUE(HasSpan('b', 'c', file_, "package")); + EXPECT_TRUE(HasSpan('d', 'e', file_, "dependency", 0)); + EXPECT_TRUE(HasSpan('f', 'g', file_, "dependency", 1)); } TEST_F(SourceInfoTest, Messages) { EXPECT_TRUE(Parse( - "/*a*/message /*b*/Foo/*c*/ {}/*d*/\n" - "/*e*/message /*f*/Bar/*g*/ {}/*h*/\n")); + "$a$message $b$Foo$c$ {}$d$\n" + "$e$message $f$Bar$g$ {}$h$\n")); - EXPECT_TRUE(HasSpan("a", "d", file_.message_type(0))); - EXPECT_TRUE(HasSpan("b", "c", file_.message_type(0), "name")); - EXPECT_TRUE(HasSpan("e", "h", file_.message_type(1))); - EXPECT_TRUE(HasSpan("f", "g", file_.message_type(1), "name")); + EXPECT_TRUE(HasSpan('a', 'd', file_.message_type(0))); + EXPECT_TRUE(HasSpan('b', 'c', file_.message_type(0), "name")); + EXPECT_TRUE(HasSpan('e', 'h', file_.message_type(1))); + EXPECT_TRUE(HasSpan('f', 'g', file_.message_type(1), "name")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1592,24 +1723,24 @@ TEST_F(SourceInfoTest, Messages) { TEST_F(SourceInfoTest, Fields) { EXPECT_TRUE(Parse( "message Foo {\n" - " /*a*/optional/*b*/ /*c*/int32/*d*/ /*e*/bar/*f*/ = /*g*/1/*h*/;/*i*/\n" - " /*j*/repeated/*k*/ /*l*/X.Y/*m*/ /*n*/baz/*o*/ = /*p*/2/*q*/;/*r*/\n" + " $a$optional$b$ $c$int32$d$ $e$bar$f$ = $g$1$h$;$i$\n" + " $j$repeated$k$ $l$X.Y$m$ $n$baz$o$ = $p$2$q$;$r$\n" "}\n")); const FieldDescriptorProto& field1 = file_.message_type(0).field(0); const FieldDescriptorProto& field2 = file_.message_type(0).field(1); - EXPECT_TRUE(HasSpan("a", "i", field1)); - EXPECT_TRUE(HasSpan("a", "b", field1, "label")); - EXPECT_TRUE(HasSpan("c", "d", field1, "type")); - EXPECT_TRUE(HasSpan("e", "f", field1, "name")); - EXPECT_TRUE(HasSpan("g", "h", field1, "number")); + EXPECT_TRUE(HasSpan('a', 'i', field1)); + EXPECT_TRUE(HasSpan('a', 'b', field1, "label")); + EXPECT_TRUE(HasSpan('c', 'd', field1, "type")); + EXPECT_TRUE(HasSpan('e', 'f', field1, "name")); + EXPECT_TRUE(HasSpan('g', 'h', field1, "number")); - EXPECT_TRUE(HasSpan("j", "r", field2)); - EXPECT_TRUE(HasSpan("j", "k", field2, "label")); - EXPECT_TRUE(HasSpan("l", "m", field2, "type_name")); - EXPECT_TRUE(HasSpan("n", "o", field2, "name")); - EXPECT_TRUE(HasSpan("p", "q", field2, "number")); + EXPECT_TRUE(HasSpan('j', 'r', field2)); + EXPECT_TRUE(HasSpan('j', 'k', field2, "label")); + EXPECT_TRUE(HasSpan('l', 'm', field2, "type_name")); + EXPECT_TRUE(HasSpan('n', 'o', field2, "name")); + EXPECT_TRUE(HasSpan('p', 'q', field2, "number")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1619,31 +1750,31 @@ TEST_F(SourceInfoTest, Fields) { TEST_F(SourceInfoTest, Extensions) { EXPECT_TRUE(Parse( - "/*a*/extend /*b*/Foo/*c*/ {\n" - " /*d*/optional/*e*/ int32 bar = 1;/*f*/\n" - " /*g*/repeated/*h*/ X.Y baz = 2;/*i*/\n" - "}/*j*/\n" - "/*k*/extend /*l*/Bar/*m*/ {\n" - " /*n*/optional int32 qux = 1;/*o*/\n" - "}/*p*/\n")); + "$a$extend $b$Foo$c$ {\n" + " $d$optional$e$ int32 bar = 1;$f$\n" + " $g$repeated$h$ X.Y baz = 2;$i$\n" + "}$j$\n" + "$k$extend $l$Bar$m$ {\n" + " $n$optional int32 qux = 1;$o$\n" + "}$p$\n")); const FieldDescriptorProto& field1 = file_.extension(0); const FieldDescriptorProto& field2 = file_.extension(1); const FieldDescriptorProto& field3 = file_.extension(2); - EXPECT_TRUE(HasSpan("a", "j", file_, "extension")); - EXPECT_TRUE(HasSpan("k", "p", file_, "extension")); + EXPECT_TRUE(HasSpan('a', 'j', file_, "extension")); + EXPECT_TRUE(HasSpan('k', 'p', file_, "extension")); - EXPECT_TRUE(HasSpan("d", "f", field1)); - EXPECT_TRUE(HasSpan("d", "e", field1, "label")); - EXPECT_TRUE(HasSpan("b", "c", field1, "extendee")); + EXPECT_TRUE(HasSpan('d', 'f', field1)); + EXPECT_TRUE(HasSpan('d', 'e', field1, "label")); + EXPECT_TRUE(HasSpan('b', 'c', field1, "extendee")); - EXPECT_TRUE(HasSpan("g", "i", field2)); - EXPECT_TRUE(HasSpan("g", "h", field2, "label")); - EXPECT_TRUE(HasSpan("b", "c", field2, "extendee")); + EXPECT_TRUE(HasSpan('g', 'i', field2)); + EXPECT_TRUE(HasSpan('g', 'h', field2, "label")); + EXPECT_TRUE(HasSpan('b', 'c', field2, "extendee")); - EXPECT_TRUE(HasSpan("n", "o", field3)); - EXPECT_TRUE(HasSpan("l", "m", field3, "extendee")); + EXPECT_TRUE(HasSpan('n', 'o', field3)); + EXPECT_TRUE(HasSpan('l', 'm', field3, "extendee")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1662,32 +1793,32 @@ TEST_F(SourceInfoTest, Extensions) { TEST_F(SourceInfoTest, NestedExtensions) { EXPECT_TRUE(Parse( "message Message {\n" - " /*a*/extend /*b*/Foo/*c*/ {\n" - " /*d*/optional/*e*/ int32 bar = 1;/*f*/\n" - " /*g*/repeated/*h*/ X.Y baz = 2;/*i*/\n" - " }/*j*/\n" - " /*k*/extend /*l*/Bar/*m*/ {\n" - " /*n*/optional int32 qux = 1;/*o*/\n" - " }/*p*/\n" + " $a$extend $b$Foo$c$ {\n" + " $d$optional$e$ int32 bar = 1;$f$\n" + " $g$repeated$h$ X.Y baz = 2;$i$\n" + " }$j$\n" + " $k$extend $l$Bar$m$ {\n" + " $n$optional int32 qux = 1;$o$\n" + " }$p$\n" "}\n")); const FieldDescriptorProto& field1 = file_.message_type(0).extension(0); const FieldDescriptorProto& field2 = file_.message_type(0).extension(1); const FieldDescriptorProto& field3 = file_.message_type(0).extension(2); - EXPECT_TRUE(HasSpan("a", "j", file_.message_type(0), "extension")); - EXPECT_TRUE(HasSpan("k", "p", file_.message_type(0), "extension")); + EXPECT_TRUE(HasSpan('a', 'j', file_.message_type(0), "extension")); + EXPECT_TRUE(HasSpan('k', 'p', file_.message_type(0), "extension")); - EXPECT_TRUE(HasSpan("d", "f", field1)); - EXPECT_TRUE(HasSpan("d", "e", field1, "label")); - EXPECT_TRUE(HasSpan("b", "c", field1, "extendee")); + EXPECT_TRUE(HasSpan('d', 'f', field1)); + EXPECT_TRUE(HasSpan('d', 'e', field1, "label")); + EXPECT_TRUE(HasSpan('b', 'c', field1, "extendee")); - EXPECT_TRUE(HasSpan("g", "i", field2)); - EXPECT_TRUE(HasSpan("g", "h", field2, "label")); - EXPECT_TRUE(HasSpan("b", "c", field2, "extendee")); + EXPECT_TRUE(HasSpan('g', 'i', field2)); + EXPECT_TRUE(HasSpan('g', 'h', field2, "label")); + EXPECT_TRUE(HasSpan('b', 'c', field2, "extendee")); - EXPECT_TRUE(HasSpan("n", "o", field3)); - EXPECT_TRUE(HasSpan("l", "m", field3, "extendee")); + EXPECT_TRUE(HasSpan('n', 'o', field3)); + EXPECT_TRUE(HasSpan('l', 'm', field3, "extendee")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1708,8 +1839,8 @@ TEST_F(SourceInfoTest, NestedExtensions) { TEST_F(SourceInfoTest, ExtensionRanges) { EXPECT_TRUE(Parse( "message Message {\n" - " /*a*/extensions /*b*/1/*c*/ to /*d*/4/*e*/, /*f*/6/*g*/;/*h*/\n" - " /*i*/extensions /*j*/8/*k*/ to /*l*/max/*m*/;/*n*/\n" + " $a$extensions $b$1$c$ to $d$4$e$, $f$6$g$;$h$\n" + " $i$extensions $j$8$k$ to $l$max$m$;$n$\n" "}\n")); const DescriptorProto::ExtensionRange& range1 = @@ -1719,20 +1850,20 @@ TEST_F(SourceInfoTest, ExtensionRanges) { const DescriptorProto::ExtensionRange& range3 = file_.message_type(0).extension_range(2); - EXPECT_TRUE(HasSpan("a", "h", file_.message_type(0), "extension_range")); - EXPECT_TRUE(HasSpan("i", "n", file_.message_type(0), "extension_range")); + EXPECT_TRUE(HasSpan('a', 'h', file_.message_type(0), "extension_range")); + EXPECT_TRUE(HasSpan('i', 'n', file_.message_type(0), "extension_range")); - EXPECT_TRUE(HasSpan("b", "e", range1)); - EXPECT_TRUE(HasSpan("b", "c", range1, "start")); - EXPECT_TRUE(HasSpan("d", "e", range1, "end")); + EXPECT_TRUE(HasSpan('b', 'e', range1)); + EXPECT_TRUE(HasSpan('b', 'c', range1, "start")); + EXPECT_TRUE(HasSpan('d', 'e', range1, "end")); - EXPECT_TRUE(HasSpan("f", "g", range2)); - EXPECT_TRUE(HasSpan("f", "g", range2, "start")); - EXPECT_TRUE(HasSpan("f", "g", range2, "end")); + EXPECT_TRUE(HasSpan('f', 'g', range2)); + EXPECT_TRUE(HasSpan('f', 'g', range2, "start")); + EXPECT_TRUE(HasSpan('f', 'g', range2, "end")); - EXPECT_TRUE(HasSpan("j", "m", range3)); - EXPECT_TRUE(HasSpan("j", "k", range3, "start")); - EXPECT_TRUE(HasSpan("l", "m", range3, "end")); + EXPECT_TRUE(HasSpan('j', 'm', range3)); + EXPECT_TRUE(HasSpan('j', 'k', range3, "start")); + EXPECT_TRUE(HasSpan('l', 'm', range3, "end")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1743,22 +1874,22 @@ TEST_F(SourceInfoTest, ExtensionRanges) { TEST_F(SourceInfoTest, NestedMessages) { EXPECT_TRUE(Parse( "message Foo {\n" - " /*a*/message /*b*/Bar/*c*/ {\n" - " /*d*/message /*e*/Baz/*f*/ {}/*g*/\n" - " }/*h*/\n" - " /*i*/message /*j*/Qux/*k*/ {}/*l*/\n" + " $a$message $b$Bar$c$ {\n" + " $d$message $e$Baz$f$ {}$g$\n" + " }$h$\n" + " $i$message $j$Qux$k$ {}$l$\n" "}\n")); const DescriptorProto& bar = file_.message_type(0).nested_type(0); const DescriptorProto& baz = bar.nested_type(0); const DescriptorProto& qux = file_.message_type(0).nested_type(1); - EXPECT_TRUE(HasSpan("a", "h", bar)); - EXPECT_TRUE(HasSpan("b", "c", bar, "name")); - EXPECT_TRUE(HasSpan("d", "g", baz)); - EXPECT_TRUE(HasSpan("e", "f", baz, "name")); - EXPECT_TRUE(HasSpan("i", "l", qux)); - EXPECT_TRUE(HasSpan("j", "k", qux, "name")); + EXPECT_TRUE(HasSpan('a', 'h', bar)); + EXPECT_TRUE(HasSpan('b', 'c', bar, "name")); + EXPECT_TRUE(HasSpan('d', 'g', baz)); + EXPECT_TRUE(HasSpan('e', 'f', baz, "name")); + EXPECT_TRUE(HasSpan('i', 'l', qux)); + EXPECT_TRUE(HasSpan('j', 'k', qux, "name")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1770,9 +1901,9 @@ TEST_F(SourceInfoTest, Groups) { EXPECT_TRUE(Parse( "message Foo {\n" " message Bar {}\n" - " /*a*/optional/*b*/ /*c*/group/*d*/ /*e*/Baz/*f*/ = /*g*/1/*h*/ {\n" - " /*i*/message Qux {}/*j*/\n" - " }/*k*/\n" + " $a$optional$b$ $c$group$d$ $e$Baz$f$ = $g$1$h$ {\n" + " $i$message Qux {}$j$\n" + " }$k$\n" "}\n")); const DescriptorProto& bar = file_.message_type(0).nested_type(0); @@ -1780,16 +1911,16 @@ TEST_F(SourceInfoTest, Groups) { const DescriptorProto& qux = baz.nested_type(0); const FieldDescriptorProto& field = file_.message_type(0).field(0); - EXPECT_TRUE(HasSpan("a", "k", field)); - EXPECT_TRUE(HasSpan("a", "b", field, "label")); - EXPECT_TRUE(HasSpan("c", "d", field, "type")); - EXPECT_TRUE(HasSpan("e", "f", field, "name")); - EXPECT_TRUE(HasSpan("e", "f", field, "type_name")); - EXPECT_TRUE(HasSpan("g", "h", field, "number")); + EXPECT_TRUE(HasSpan('a', 'k', field)); + EXPECT_TRUE(HasSpan('a', 'b', field, "label")); + EXPECT_TRUE(HasSpan('c', 'd', field, "type")); + EXPECT_TRUE(HasSpan('e', 'f', field, "name")); + EXPECT_TRUE(HasSpan('e', 'f', field, "type_name")); + EXPECT_TRUE(HasSpan('g', 'h', field, "number")); - EXPECT_TRUE(HasSpan("a", "k", baz)); - EXPECT_TRUE(HasSpan("e", "f", baz, "name")); - EXPECT_TRUE(HasSpan("i", "j", qux)); + EXPECT_TRUE(HasSpan('a', 'k', baz)); + EXPECT_TRUE(HasSpan('e', 'f', baz, "name")); + EXPECT_TRUE(HasSpan('i', 'j', qux)); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1802,13 +1933,13 @@ TEST_F(SourceInfoTest, Groups) { TEST_F(SourceInfoTest, Enums) { EXPECT_TRUE(Parse( - "/*a*/enum /*b*/Foo/*c*/ {}/*d*/\n" - "/*e*/enum /*f*/Bar/*g*/ {}/*h*/\n")); + "$a$enum $b$Foo$c$ {}$d$\n" + "$e$enum $f$Bar$g$ {}$h$\n")); - EXPECT_TRUE(HasSpan("a", "d", file_.enum_type(0))); - EXPECT_TRUE(HasSpan("b", "c", file_.enum_type(0), "name")); - EXPECT_TRUE(HasSpan("e", "h", file_.enum_type(1))); - EXPECT_TRUE(HasSpan("f", "g", file_.enum_type(1), "name")); + EXPECT_TRUE(HasSpan('a', 'd', file_.enum_type(0))); + EXPECT_TRUE(HasSpan('b', 'c', file_.enum_type(0), "name")); + EXPECT_TRUE(HasSpan('e', 'h', file_.enum_type(1))); + EXPECT_TRUE(HasSpan('f', 'g', file_.enum_type(1), "name")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1817,19 +1948,19 @@ TEST_F(SourceInfoTest, Enums) { TEST_F(SourceInfoTest, EnumValues) { EXPECT_TRUE(Parse( "enum Foo {\n" - " /*a*/BAR/*b*/ = /*c*/1/*d*/;/*e*/\n" - " /*f*/BAZ/*g*/ = /*h*/2/*i*/;/*j*/\n" + " $a$BAR$b$ = $c$1$d$;$e$\n" + " $f$BAZ$g$ = $h$2$i$;$j$\n" "}")); const EnumValueDescriptorProto& bar = file_.enum_type(0).value(0); const EnumValueDescriptorProto& baz = file_.enum_type(0).value(1); - EXPECT_TRUE(HasSpan("a", "e", bar)); - EXPECT_TRUE(HasSpan("a", "b", bar, "name")); - EXPECT_TRUE(HasSpan("c", "d", bar, "number")); - EXPECT_TRUE(HasSpan("f", "j", baz)); - EXPECT_TRUE(HasSpan("f", "g", baz, "name")); - EXPECT_TRUE(HasSpan("h", "i", baz, "number")); + EXPECT_TRUE(HasSpan('a', 'e', bar)); + EXPECT_TRUE(HasSpan('a', 'b', bar, "name")); + EXPECT_TRUE(HasSpan('c', 'd', bar, "number")); + EXPECT_TRUE(HasSpan('f', 'j', baz)); + EXPECT_TRUE(HasSpan('f', 'g', baz, "name")); + EXPECT_TRUE(HasSpan('h', 'i', baz, "number")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1840,17 +1971,17 @@ TEST_F(SourceInfoTest, EnumValues) { TEST_F(SourceInfoTest, NestedEnums) { EXPECT_TRUE(Parse( "message Foo {\n" - " /*a*/enum /*b*/Bar/*c*/ {}/*d*/\n" - " /*e*/enum /*f*/Baz/*g*/ {}/*h*/\n" + " $a$enum $b$Bar$c$ {}$d$\n" + " $e$enum $f$Baz$g$ {}$h$\n" "}\n")); const EnumDescriptorProto& bar = file_.message_type(0).enum_type(0); const EnumDescriptorProto& baz = file_.message_type(0).enum_type(1); - EXPECT_TRUE(HasSpan("a", "d", bar)); - EXPECT_TRUE(HasSpan("b", "c", bar, "name")); - EXPECT_TRUE(HasSpan("e", "h", baz)); - EXPECT_TRUE(HasSpan("f", "g", baz, "name")); + EXPECT_TRUE(HasSpan('a', 'd', bar)); + EXPECT_TRUE(HasSpan('b', 'c', bar, "name")); + EXPECT_TRUE(HasSpan('e', 'h', baz)); + EXPECT_TRUE(HasSpan('f', 'g', baz, "name")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1860,37 +1991,37 @@ TEST_F(SourceInfoTest, NestedEnums) { TEST_F(SourceInfoTest, Services) { EXPECT_TRUE(Parse( - "/*a*/service /*b*/Foo/*c*/ {}/*d*/\n" - "/*e*/service /*f*/Bar/*g*/ {}/*h*/\n")); + "$a$service $b$Foo$c$ {}$d$\n" + "$e$service $f$Bar$g$ {}$h$\n")); - EXPECT_TRUE(HasSpan("a", "d", file_.service(0))); - EXPECT_TRUE(HasSpan("b", "c", file_.service(0), "name")); - EXPECT_TRUE(HasSpan("e", "h", file_.service(1))); - EXPECT_TRUE(HasSpan("f", "g", file_.service(1), "name")); + EXPECT_TRUE(HasSpan('a', 'd', file_.service(0))); + EXPECT_TRUE(HasSpan('b', 'c', file_.service(0), "name")); + EXPECT_TRUE(HasSpan('e', 'h', file_.service(1))); + EXPECT_TRUE(HasSpan('f', 'g', file_.service(1), "name")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); } -TEST_F(SourceInfoTest, Methods) { +TEST_F(SourceInfoTest, MethodsAndStreams) { EXPECT_TRUE(Parse( "service Foo {\n" - " /*a*/rpc /*b*/Bar/*c*/(/*d*/X/*e*/) returns(/*f*/Y/*g*/);/*h*/" - " /*i*/rpc /*j*/Baz/*k*/(/*l*/Z/*m*/) returns(/*n*/W/*o*/);/*p*/" + " $a$rpc $b$Bar$c$($d$X$e$) returns($f$Y$g$);$h$" + " $i$rpc $j$Baz$k$($l$Z$m$) returns($n$W$o$);$p$" "}")); const MethodDescriptorProto& bar = file_.service(0).method(0); const MethodDescriptorProto& baz = file_.service(0).method(1); - EXPECT_TRUE(HasSpan("a", "h", bar)); - EXPECT_TRUE(HasSpan("b", "c", bar, "name")); - EXPECT_TRUE(HasSpan("d", "e", bar, "input_type")); - EXPECT_TRUE(HasSpan("f", "g", bar, "output_type")); + EXPECT_TRUE(HasSpan('a', 'h', bar)); + EXPECT_TRUE(HasSpan('b', 'c', bar, "name")); + EXPECT_TRUE(HasSpan('d', 'e', bar, "input_type")); + EXPECT_TRUE(HasSpan('f', 'g', bar, "output_type")); - EXPECT_TRUE(HasSpan("i", "p", baz)); - EXPECT_TRUE(HasSpan("j", "k", baz, "name")); - EXPECT_TRUE(HasSpan("l", "m", baz, "input_type")); - EXPECT_TRUE(HasSpan("n", "o", baz, "output_type")); + EXPECT_TRUE(HasSpan('i', 'p', baz)); + EXPECT_TRUE(HasSpan('j', 'k', baz, "name")); + EXPECT_TRUE(HasSpan('l', 'm', baz, "input_type")); + EXPECT_TRUE(HasSpan('n', 'o', baz, "output_type")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1900,13 +2031,13 @@ TEST_F(SourceInfoTest, Methods) { TEST_F(SourceInfoTest, Options) { EXPECT_TRUE(Parse( - "/*a*/option /*b*/foo/*c*/./*d*/(/*e*/bar.baz/*f*/)/*g*/ = " - "/*h*/123/*i*/;/*j*/\n" - "/*k*/option qux = /*l*/-123/*m*/;/*n*/\n" - "/*o*/option corge = /*p*/abc/*q*/;/*r*/\n" - "/*s*/option grault = /*t*/'blah'/*u*/;/*v*/\n" - "/*w*/option garply = /*x*/{ yadda yadda }/*y*/;/*z*/\n" - "/*0*/option waldo = /*1*/123.0/*2*/;/*3*/\n" + "$a$option $b$foo$c$.$d$($e$bar.baz$f$)$g$ = " + "$h$123$i$;$j$\n" + "$k$option qux = $l$-123$m$;$n$\n" + "$o$option corge = $p$abc$q$;$r$\n" + "$s$option grault = $t$'blah'$u$;$v$\n" + "$w$option garply = $x${ yadda yadda }$y$;$z$\n" + "$0$option waldo = $1$123.0$2$;$3$\n" )); const UninterpretedOption& option1 = file_.options().uninterpreted_option(0); @@ -1916,29 +2047,29 @@ TEST_F(SourceInfoTest, Options) { const UninterpretedOption& option5 = file_.options().uninterpreted_option(4); const UninterpretedOption& option6 = file_.options().uninterpreted_option(5); - EXPECT_TRUE(HasSpan("a", "j", file_.options())); - EXPECT_TRUE(HasSpan("b", "i", option1)); - EXPECT_TRUE(HasSpan("b", "g", option1, "name")); - EXPECT_TRUE(HasSpan("b", "c", option1.name(0))); - EXPECT_TRUE(HasSpan("b", "c", option1.name(0), "name_part")); - EXPECT_TRUE(HasSpan("d", "g", option1.name(1))); - EXPECT_TRUE(HasSpan("e", "f", option1.name(1), "name_part")); - EXPECT_TRUE(HasSpan("h", "i", option1, "positive_int_value")); + EXPECT_TRUE(HasSpan('a', 'j', file_.options())); + EXPECT_TRUE(HasSpan('a', 'j', option1)); + EXPECT_TRUE(HasSpan('b', 'g', option1, "name")); + EXPECT_TRUE(HasSpan('b', 'c', option1.name(0))); + EXPECT_TRUE(HasSpan('b', 'c', option1.name(0), "name_part")); + EXPECT_TRUE(HasSpan('d', 'g', option1.name(1))); + EXPECT_TRUE(HasSpan('e', 'f', option1.name(1), "name_part")); + EXPECT_TRUE(HasSpan('h', 'i', option1, "positive_int_value")); - EXPECT_TRUE(HasSpan("k", "n", file_.options())); - EXPECT_TRUE(HasSpan("l", "m", option2, "negative_int_value")); + EXPECT_TRUE(HasSpan('k', 'n', file_.options())); + EXPECT_TRUE(HasSpan('l', 'm', option2, "negative_int_value")); - EXPECT_TRUE(HasSpan("o", "r", file_.options())); - EXPECT_TRUE(HasSpan("p", "q", option3, "identifier_value")); + EXPECT_TRUE(HasSpan('o', 'r', file_.options())); + EXPECT_TRUE(HasSpan('p', 'q', option3, "identifier_value")); - EXPECT_TRUE(HasSpan("s", "v", file_.options())); - EXPECT_TRUE(HasSpan("t", "u", option4, "string_value")); + EXPECT_TRUE(HasSpan('s', 'v', file_.options())); + EXPECT_TRUE(HasSpan('t', 'u', option4, "string_value")); - EXPECT_TRUE(HasSpan("w", "z", file_.options())); - EXPECT_TRUE(HasSpan("x", "y", option5, "aggregate_value")); + EXPECT_TRUE(HasSpan('w', 'z', file_.options())); + EXPECT_TRUE(HasSpan('x', 'y', option5, "aggregate_value")); - EXPECT_TRUE(HasSpan("0", "3", file_.options())); - EXPECT_TRUE(HasSpan("1", "2", option6, "double_value")); + EXPECT_TRUE(HasSpan('0', '3', file_.options())); + EXPECT_TRUE(HasSpan('1', '2', option6, "double_value")); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -1967,22 +2098,22 @@ TEST_F(SourceInfoTest, Options) { TEST_F(SourceInfoTest, ScopedOptions) { EXPECT_TRUE(Parse( "message Foo {\n" - " /*a*/option mopt = 1;/*b*/\n" + " $a$option mopt = 1;$b$\n" "}\n" "enum Bar {\n" - " /*c*/option eopt = 1;/*d*/\n" + " $c$option eopt = 1;$d$\n" "}\n" "service Baz {\n" - " /*e*/option sopt = 1;/*f*/\n" + " $e$option sopt = 1;$f$\n" " rpc M(X) returns(Y) {\n" - " /*g*/option mopt = 1;/*h*/\n" + " $g$option mopt = 1;$h$\n" " }\n" "}\n")); - EXPECT_TRUE(HasSpan("a", "b", file_.message_type(0).options())); - EXPECT_TRUE(HasSpan("c", "d", file_.enum_type(0).options())); - EXPECT_TRUE(HasSpan("e", "f", file_.service(0).options())); - EXPECT_TRUE(HasSpan("g", "h", file_.service(0).method(0).options())); + EXPECT_TRUE(HasSpan('a', 'b', file_.message_type(0).options())); + EXPECT_TRUE(HasSpan('c', 'd', file_.enum_type(0).options())); + EXPECT_TRUE(HasSpan('e', 'f', file_.service(0).options())); + EXPECT_TRUE(HasSpan('g', 'h', file_.service(0).method(0).options())); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -2045,8 +2176,8 @@ TEST_F(SourceInfoTest, FieldOptions) { EXPECT_TRUE(Parse( "message Foo {" " optional int32 bar = 1 " - "/*a*/[default=/*b*/123/*c*/,/*d*/opt1=123/*e*/," - "/*f*/opt2='hi'/*g*/]/*h*/;" + "$a$[default=$b$123$c$,$d$opt1=123$e$," + "$f$opt2='hi'$g$]$h$;" "}\n" )); @@ -2054,10 +2185,10 @@ TEST_F(SourceInfoTest, FieldOptions) { const UninterpretedOption& option1 = field.options().uninterpreted_option(0); const UninterpretedOption& option2 = field.options().uninterpreted_option(1); - EXPECT_TRUE(HasSpan("a", "h", field.options())); - EXPECT_TRUE(HasSpan("b", "c", field, "default_value")); - EXPECT_TRUE(HasSpan("d", "e", option1)); - EXPECT_TRUE(HasSpan("f", "g", option2)); + EXPECT_TRUE(HasSpan('a', 'h', field.options())); + EXPECT_TRUE(HasSpan('b', 'c', field, "default_value")); + EXPECT_TRUE(HasSpan('d', 'e', option1)); + EXPECT_TRUE(HasSpan('f', 'g', option2)); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -2084,7 +2215,7 @@ TEST_F(SourceInfoTest, EnumValueOptions) { // syntax used for enum options is understood. EXPECT_TRUE(Parse( "enum Foo {" - " BAR = 1 /*a*/[/*b*/opt1=123/*c*/,/*d*/opt2='hi'/*e*/]/*f*/;" + " BAR = 1 $a$[$b$opt1=123$c$,$d$opt2='hi'$e$]$f$;" "}\n" )); @@ -2092,9 +2223,9 @@ TEST_F(SourceInfoTest, EnumValueOptions) { const UninterpretedOption& option1 = value.options().uninterpreted_option(0); const UninterpretedOption& option2 = value.options().uninterpreted_option(1); - EXPECT_TRUE(HasSpan("a", "f", value.options())); - EXPECT_TRUE(HasSpan("b", "c", option1)); - EXPECT_TRUE(HasSpan("d", "e", option2)); + EXPECT_TRUE(HasSpan('a', 'f', value.options())); + EXPECT_TRUE(HasSpan('b', 'c', option1)); + EXPECT_TRUE(HasSpan('d', 'e', option2)); // Ignore these. EXPECT_TRUE(HasSpan(file_)); @@ -2113,6 +2244,127 @@ TEST_F(SourceInfoTest, EnumValueOptions) { EXPECT_TRUE(HasSpan(option2, "string_value")); } +TEST_F(SourceInfoTest, DocComments) { + EXPECT_TRUE(Parse( + "// Foo leading\n" + "// line 2\n" + "$a$message Foo {\n" + " // Foo trailing\n" + " // line 2\n" + "\n" + " // ignored\n" + "\n" + " // bar leading\n" + " $b$optional int32 bar = 1;$c$\n" + " // bar trailing\n" + "}$d$\n" + "// ignored\n" + )); + + const DescriptorProto& foo = file_.message_type(0); + const FieldDescriptorProto& bar = foo.field(0); + + EXPECT_TRUE(HasSpanWithComment('a', 'd', foo, + " Foo leading\n line 2\n", + " Foo trailing\n line 2\n")); + EXPECT_TRUE(HasSpanWithComment('b', 'c', bar, + " bar leading\n", + " bar trailing\n")); + + // Ignore these. + EXPECT_TRUE(HasSpan(file_)); + EXPECT_TRUE(HasSpan(foo, "name")); + EXPECT_TRUE(HasSpan(bar, "label")); + EXPECT_TRUE(HasSpan(bar, "type")); + EXPECT_TRUE(HasSpan(bar, "name")); + EXPECT_TRUE(HasSpan(bar, "number")); +} + +TEST_F(SourceInfoTest, DocComments2) { + EXPECT_TRUE(Parse( + "// ignored\n" + "syntax = \"proto2\";\n" + "// Foo leading\n" + "// line 2\n" + "$a$message Foo {\n" + " /* Foo trailing\n" + " * line 2 */\n" + " // ignored\n" + " /* bar leading\n" + " */" + " $b$optional int32 bar = 1;$c$ // bar trailing\n" + " // ignored\n" + "}$d$\n" + "// ignored\n" + "\n" + "// option leading\n" + "$e$option baz = 123;$f$\n" + "// option trailing\n" + )); + + const DescriptorProto& foo = file_.message_type(0); + const FieldDescriptorProto& bar = foo.field(0); + const UninterpretedOption& baz = file_.options().uninterpreted_option(0); + + EXPECT_TRUE(HasSpanWithComment('a', 'd', foo, + " Foo leading\n line 2\n", + " Foo trailing\n line 2 ")); + EXPECT_TRUE(HasSpanWithComment('b', 'c', bar, + " bar leading\n", + " bar trailing\n")); + EXPECT_TRUE(HasSpanWithComment('e', 'f', baz, + " option leading\n", + " option trailing\n")); + + // Ignore these. + EXPECT_TRUE(HasSpan(file_)); + EXPECT_TRUE(HasSpan(foo, "name")); + EXPECT_TRUE(HasSpan(bar, "label")); + EXPECT_TRUE(HasSpan(bar, "type")); + EXPECT_TRUE(HasSpan(bar, "name")); + EXPECT_TRUE(HasSpan(bar, "number")); + EXPECT_TRUE(HasSpan(file_.options())); + EXPECT_TRUE(HasSpan(baz, "name")); + EXPECT_TRUE(HasSpan(baz.name(0))); + EXPECT_TRUE(HasSpan(baz.name(0), "name_part")); + EXPECT_TRUE(HasSpan(baz, "positive_int_value")); +} + +TEST_F(SourceInfoTest, DocComments3) { + EXPECT_TRUE(Parse( + "$a$message Foo {\n" + " // bar leading\n" + " $b$optional int32 bar = 1 [(baz.qux) = {}];$c$\n" + " // bar trailing\n" + "}$d$\n" + "// ignored\n" + )); + + const DescriptorProto& foo = file_.message_type(0); + const FieldDescriptorProto& bar = foo.field(0); + + EXPECT_TRUE(HasSpanWithComment('b', 'c', bar, + " bar leading\n", + " bar trailing\n")); + + // Ignore these. + EXPECT_TRUE(HasSpan(file_)); + EXPECT_TRUE(HasSpan(foo)); + EXPECT_TRUE(HasSpan(foo, "name")); + EXPECT_TRUE(HasSpan(bar, "label")); + EXPECT_TRUE(HasSpan(bar, "type")); + EXPECT_TRUE(HasSpan(bar, "name")); + EXPECT_TRUE(HasSpan(bar, "number")); + EXPECT_TRUE(HasSpan(bar.options())); + EXPECT_TRUE(HasSpan(bar.options().uninterpreted_option(0))); + EXPECT_TRUE(HasSpan(bar.options().uninterpreted_option(0), "name")); + EXPECT_TRUE(HasSpan(bar.options().uninterpreted_option(0).name(0))); + EXPECT_TRUE(HasSpan( + bar.options().uninterpreted_option(0).name(0), "name_part")); + EXPECT_TRUE(HasSpan( + bar.options().uninterpreted_option(0), "aggregate_value")); +} + // =================================================================== } // anonymous namespace diff --git a/third_party/protobuf/src/google/protobuf/compiler/plugin.h b/third_party/protobuf/src/google/protobuf/compiler/plugin.h index 7c40333..6fa2de1 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/plugin.h +++ b/third_party/protobuf/src/google/protobuf/compiler/plugin.h @@ -56,7 +56,6 @@ #define GOOGLE_PROTOBUF_COMPILER_PLUGIN_H__ #include <google/protobuf/stubs/common.h> - namespace google { namespace protobuf { namespace compiler { diff --git a/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.cc b/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.cc index c956188..7b0b1f9 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.cc @@ -1,14 +1,17 @@ // Generated by the protocol buffer compiler. DO NOT EDIT! +// source: google/protobuf/compiler/plugin.proto #define INTERNAL_SUPPRESS_PROTOBUF_FIELD_DEPRECATION #include "google/protobuf/compiler/plugin.pb.h" #include <algorithm> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/stubs/once.h> #include <google/protobuf/io/coded_stream.h> #include <google/protobuf/wire_format_lite_inl.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/generated_message_reflection.h> #include <google/protobuf/reflection_ops.h> #include <google/protobuf/wire_format.h> // @@protoc_insertion_point(includes) @@ -124,6 +127,7 @@ void protobuf_AddDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto() { if (already_here) return; already_here = true; GOOGLE_PROTOBUF_VERIFY_VERSION; + ::google::protobuf::protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); ::google::protobuf::DescriptorPool::InternalAddGeneratedFile( "\n%google/protobuf/compiler/plugin.proto\022" @@ -253,7 +257,7 @@ bool CodeGeneratorRequest::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_parameter; break; } - + // optional string parameter = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -270,7 +274,7 @@ bool CodeGeneratorRequest::MergePartialFromCodedStream( if (input->ExpectTag(122)) goto parse_proto_file; break; } - + // repeated .google.protobuf.FileDescriptorProto proto_file = 15; case 15: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -285,14 +289,15 @@ bool CodeGeneratorRequest::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -311,7 +316,7 @@ void CodeGeneratorRequest::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->file_to_generate(i), output); } - + // optional string parameter = 2; if (has_parameter()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -320,13 +325,13 @@ void CodeGeneratorRequest::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 2, this->parameter(), output); } - + // repeated .google.protobuf.FileDescriptorProto proto_file = 15; for (int i = 0; i < this->proto_file_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 15, this->proto_file(i), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -343,7 +348,7 @@ void CodeGeneratorRequest::SerializeWithCachedSizes( target = ::google::protobuf::internal::WireFormatLite:: WriteStringToArray(1, this->file_to_generate(i), target); } - + // optional string parameter = 2; if (has_parameter()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -353,14 +358,14 @@ void CodeGeneratorRequest::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 2, this->parameter(), target); } - + // repeated .google.protobuf.FileDescriptorProto proto_file = 15; for (int i = 0; i < this->proto_file_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 15, this->proto_file(i), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -370,7 +375,7 @@ void CodeGeneratorRequest::SerializeWithCachedSizes( int CodeGeneratorRequest::ByteSize() const { int total_size = 0; - + if (_has_bits_[1 / 32] & (0xffu << (1 % 32))) { // optional string parameter = 2; if (has_parameter()) { @@ -378,7 +383,7 @@ int CodeGeneratorRequest::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->parameter()); } - + } // repeated string file_to_generate = 1; total_size += 1 * this->file_to_generate_size(); @@ -386,7 +391,7 @@ int CodeGeneratorRequest::ByteSize() const { total_size += ::google::protobuf::internal::WireFormatLite::StringSize( this->file_to_generate(i)); } - + // repeated .google.protobuf.FileDescriptorProto proto_file = 15; total_size += 1 * this->proto_file_size(); for (int i = 0; i < this->proto_file_size(); i++) { @@ -394,7 +399,7 @@ int CodeGeneratorRequest::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->proto_file(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -443,7 +448,7 @@ void CodeGeneratorRequest::CopyFrom(const CodeGeneratorRequest& from) { } bool CodeGeneratorRequest::IsInitialized() const { - + for (int i = 0; i < proto_file_size(); i++) { if (!this->proto_file(i).IsInitialized()) return false; } @@ -582,7 +587,7 @@ bool CodeGeneratorResponse_File::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_insertion_point; break; } - + // optional string insertion_point = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -599,7 +604,7 @@ bool CodeGeneratorResponse_File::MergePartialFromCodedStream( if (input->ExpectTag(122)) goto parse_content; break; } - + // optional string content = 15; case 15: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -616,14 +621,15 @@ bool CodeGeneratorResponse_File::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -642,7 +648,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // optional string insertion_point = 2; if (has_insertion_point()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -651,7 +657,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 2, this->insertion_point(), output); } - + // optional string content = 15; if (has_content()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -660,7 +666,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 15, this->content(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -678,7 +684,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // optional string insertion_point = 2; if (has_insertion_point()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -688,7 +694,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 2, this->insertion_point(), target); } - + // optional string content = 15; if (has_content()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -698,7 +704,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 15, this->content(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -708,7 +714,7 @@ void CodeGeneratorResponse_File::SerializeWithCachedSizes( int CodeGeneratorResponse_File::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -716,21 +722,21 @@ int CodeGeneratorResponse_File::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional string insertion_point = 2; if (has_insertion_point()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->insertion_point()); } - + // optional string content = 15; if (has_content()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->content()); } - + } if (!unknown_fields().empty()) { total_size += @@ -784,7 +790,7 @@ void CodeGeneratorResponse_File::CopyFrom(const CodeGeneratorResponse_File& from } bool CodeGeneratorResponse_File::IsInitialized() const { - + return true; } @@ -902,7 +908,7 @@ bool CodeGeneratorResponse::MergePartialFromCodedStream( if (input->ExpectTag(122)) goto parse_file; break; } - + // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; case 15: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -917,14 +923,15 @@ bool CodeGeneratorResponse::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -943,13 +950,13 @@ void CodeGeneratorResponse::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->error(), output); } - + // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; for (int i = 0; i < this->file_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 15, this->file(i), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -967,14 +974,14 @@ void CodeGeneratorResponse::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->error(), target); } - + // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; for (int i = 0; i < this->file_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 15, this->file(i), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -984,7 +991,7 @@ void CodeGeneratorResponse::SerializeWithCachedSizes( int CodeGeneratorResponse::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string error = 1; if (has_error()) { @@ -992,7 +999,7 @@ int CodeGeneratorResponse::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->error()); } - + } // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; total_size += 1 * this->file_size(); @@ -1001,7 +1008,7 @@ int CodeGeneratorResponse::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->file(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -1049,7 +1056,7 @@ void CodeGeneratorResponse::CopyFrom(const CodeGeneratorResponse& from) { } bool CodeGeneratorResponse::IsInitialized() const { - + return true; } diff --git a/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.h b/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.h index 64246fc..46478c7 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.h +++ b/third_party/protobuf/src/google/protobuf/compiler/plugin.pb.h @@ -20,10 +20,10 @@ #endif #include <google/protobuf/generated_message_util.h> +#include <google/protobuf/message.h> #include <google/protobuf/repeated_field.h> #include <google/protobuf/extension_set.h> #include <google/protobuf/unknown_field_set.h> -#include <google/protobuf/generated_message_reflection.h> #include "google/protobuf/descriptor.pb.h" // @@protoc_insertion_point(includes) @@ -46,29 +46,29 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message public: CodeGeneratorRequest(); virtual ~CodeGeneratorRequest(); - + CodeGeneratorRequest(const CodeGeneratorRequest& from); - + inline CodeGeneratorRequest& operator=(const CodeGeneratorRequest& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const CodeGeneratorRequest& default_instance(); - + void Swap(CodeGeneratorRequest* other); - + // implements Message ---------------------------------------------- - + CodeGeneratorRequest* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -76,7 +76,7 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message void MergeFrom(const CodeGeneratorRequest& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -89,13 +89,13 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated string file_to_generate = 1; inline int file_to_generate_size() const; inline void clear_file_to_generate(); @@ -111,7 +111,7 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message inline void add_file_to_generate(const char* value, size_t size); inline const ::google::protobuf::RepeatedPtrField< ::std::string>& file_to_generate() const; inline ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_file_to_generate(); - + // optional string parameter = 2; inline bool has_parameter() const; inline void clear_parameter(); @@ -122,7 +122,8 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message inline void set_parameter(const char* value, size_t size); inline ::std::string* mutable_parameter(); inline ::std::string* release_parameter(); - + inline void set_allocated_parameter(::std::string* parameter); + // repeated .google.protobuf.FileDescriptorProto proto_file = 15; inline int proto_file_size() const; inline void clear_proto_file(); @@ -134,25 +135,25 @@ class LIBPROTOC_EXPORT CodeGeneratorRequest : public ::google::protobuf::Message proto_file() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::FileDescriptorProto >* mutable_proto_file(); - + // @@protoc_insertion_point(class_scope:google.protobuf.compiler.CodeGeneratorRequest) private: inline void set_has_parameter(); inline void clear_has_parameter(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::std::string> file_to_generate_; ::std::string* parameter_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::FileDescriptorProto > proto_file_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOC_EXPORT protobuf_AddDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); - + void InitAsDefaultInstance(); static CodeGeneratorRequest* default_instance_; }; @@ -162,29 +163,29 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M public: CodeGeneratorResponse_File(); virtual ~CodeGeneratorResponse_File(); - + CodeGeneratorResponse_File(const CodeGeneratorResponse_File& from); - + inline CodeGeneratorResponse_File& operator=(const CodeGeneratorResponse_File& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const CodeGeneratorResponse_File& default_instance(); - + void Swap(CodeGeneratorResponse_File* other); - + // implements Message ---------------------------------------------- - + CodeGeneratorResponse_File* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -192,7 +193,7 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M void MergeFrom(const CodeGeneratorResponse_File& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -205,13 +206,13 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -222,7 +223,8 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // optional string insertion_point = 2; inline bool has_insertion_point() const; inline void clear_insertion_point(); @@ -233,7 +235,8 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M inline void set_insertion_point(const char* value, size_t size); inline ::std::string* mutable_insertion_point(); inline ::std::string* release_insertion_point(); - + inline void set_allocated_insertion_point(::std::string* insertion_point); + // optional string content = 15; inline bool has_content() const; inline void clear_content(); @@ -244,7 +247,8 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M inline void set_content(const char* value, size_t size); inline ::std::string* mutable_content(); inline ::std::string* release_content(); - + inline void set_allocated_content(::std::string* content); + // @@protoc_insertion_point(class_scope:google.protobuf.compiler.CodeGeneratorResponse.File) private: inline void set_has_name(); @@ -253,20 +257,20 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse_File : public ::google::protobuf::M inline void clear_has_insertion_point(); inline void set_has_content(); inline void clear_has_content(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::std::string* insertion_point_; ::std::string* content_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOC_EXPORT protobuf_AddDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); - + void InitAsDefaultInstance(); static CodeGeneratorResponse_File* default_instance_; }; @@ -276,29 +280,29 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse : public ::google::protobuf::Messag public: CodeGeneratorResponse(); virtual ~CodeGeneratorResponse(); - + CodeGeneratorResponse(const CodeGeneratorResponse& from); - + inline CodeGeneratorResponse& operator=(const CodeGeneratorResponse& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const CodeGeneratorResponse& default_instance(); - + void Swap(CodeGeneratorResponse* other); - + // implements Message ---------------------------------------------- - + CodeGeneratorResponse* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -306,7 +310,7 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse : public ::google::protobuf::Messag void MergeFrom(const CodeGeneratorResponse& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -319,15 +323,15 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse : public ::google::protobuf::Messag void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef CodeGeneratorResponse_File File; - + // accessors ------------------------------------------------------- - + // optional string error = 1; inline bool has_error() const; inline void clear_error(); @@ -338,7 +342,8 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse : public ::google::protobuf::Messag inline void set_error(const char* value, size_t size); inline ::std::string* mutable_error(); inline ::std::string* release_error(); - + inline void set_allocated_error(::std::string* error); + // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; inline int file_size() const; inline void clear_file(); @@ -350,24 +355,24 @@ class LIBPROTOC_EXPORT CodeGeneratorResponse : public ::google::protobuf::Messag file() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::compiler::CodeGeneratorResponse_File >* mutable_file(); - + // @@protoc_insertion_point(class_scope:google.protobuf.compiler.CodeGeneratorResponse) private: inline void set_has_error(); inline void clear_has_error(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* error_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::compiler::CodeGeneratorResponse_File > file_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(2 + 31) / 32]; - + friend void LIBPROTOC_EXPORT protobuf_AddDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fcompiler_2fplugin_2eproto(); - + void InitAsDefaultInstance(); static CodeGeneratorResponse* default_instance_; }; @@ -479,6 +484,18 @@ inline ::std::string* CodeGeneratorRequest::release_parameter() { return temp; } } +inline void CodeGeneratorRequest::set_allocated_parameter(::std::string* parameter) { + if (parameter_ != &::google::protobuf::internal::kEmptyString) { + delete parameter_; + } + if (parameter) { + set_has_parameter(); + parameter_ = parameter; + } else { + clear_has_parameter(); + parameter_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated .google.protobuf.FileDescriptorProto proto_file = 15; inline int CodeGeneratorRequest::proto_file_size() const { @@ -566,6 +583,18 @@ inline ::std::string* CodeGeneratorResponse_File::release_name() { return temp; } } +inline void CodeGeneratorResponse_File::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string insertion_point = 2; inline bool CodeGeneratorResponse_File::has_insertion_point() const { @@ -624,6 +653,18 @@ inline ::std::string* CodeGeneratorResponse_File::release_insertion_point() { return temp; } } +inline void CodeGeneratorResponse_File::set_allocated_insertion_point(::std::string* insertion_point) { + if (insertion_point_ != &::google::protobuf::internal::kEmptyString) { + delete insertion_point_; + } + if (insertion_point) { + set_has_insertion_point(); + insertion_point_ = insertion_point; + } else { + clear_has_insertion_point(); + insertion_point_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string content = 15; inline bool CodeGeneratorResponse_File::has_content() const { @@ -682,6 +723,18 @@ inline ::std::string* CodeGeneratorResponse_File::release_content() { return temp; } } +inline void CodeGeneratorResponse_File::set_allocated_content(::std::string* content) { + if (content_ != &::google::protobuf::internal::kEmptyString) { + delete content_; + } + if (content) { + set_has_content(); + content_ = content; + } else { + clear_has_content(); + content_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // ------------------------------------------------------------------- @@ -744,6 +797,18 @@ inline ::std::string* CodeGeneratorResponse::release_error() { return temp; } } +inline void CodeGeneratorResponse::set_allocated_error(::std::string* error) { + if (error_ != &::google::protobuf::internal::kEmptyString) { + delete error_; + } + if (error) { + set_has_error(); + error_ = error; + } else { + clear_has_error(); + error_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated .google.protobuf.compiler.CodeGeneratorResponse.File file = 15; inline int CodeGeneratorResponse::file_size() const { diff --git a/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.cc b/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.cc index 9b10937..211ac70 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.cc @@ -52,6 +52,7 @@ #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/stubs/stringprintf.h> #include <google/protobuf/io/printer.h> #include <google/protobuf/descriptor.h> #include <google/protobuf/io/zero_copy_stream.h> @@ -106,6 +107,12 @@ string NamePrefixedWithNestedTypes(const DescriptorT& descriptor, const char kDescriptorKey[] = "DESCRIPTOR"; +// Does the file have top-level enums? +inline bool HasTopLevelEnums(const FileDescriptor *file) { + return file->enum_type_count() > 0; +} + + // Should we generate generic services for this file? inline bool HasGenericServices(const FileDescriptor *file) { return file->service_count() > 0 && @@ -120,13 +127,21 @@ void PrintTopBoilerplate( // TODO(robinson): Allow parameterization of Python version? printer->Print( "# Generated by the protocol buffer compiler. DO NOT EDIT!\n" - "\n" - "from google.protobuf import descriptor\n" - "from google.protobuf import message\n" - "from google.protobuf import reflection\n"); + "# source: $filename$\n" + "\n", + "filename", file->name()); + if (HasTopLevelEnums(file)) { + printer->Print( + "from google.protobuf.internal import enum_type_wrapper\n"); + } + printer->Print( + "from google.protobuf import descriptor as _descriptor\n" + "from google.protobuf import message as _message\n" + "from google.protobuf import reflection as _reflection\n" + ); if (HasGenericServices(file)) { printer->Print( - "from google.protobuf import service\n" + "from google.protobuf import service as _service\n" "from google.protobuf import service_reflection\n"); } @@ -204,12 +219,12 @@ string StringifyDefaultValue(const FieldDescriptor& field) { case FieldDescriptor::CPPTYPE_STRING: if (field.type() == FieldDescriptor::TYPE_STRING) { return "unicode(\"" + CEscape(field.default_value_string()) + - "\", \"utf-8\")"; + "\", \"utf-8\")"; } else { return "\"" + CEscape(field.default_value_string()) + "\""; } - case FieldDescriptor::CPPTYPE_MESSAGE: - return "None"; + case FieldDescriptor::CPPTYPE_MESSAGE: + return "None"; } // (We could add a default case above but then we wouldn't get the nice // compiler warning when a new type is added.) @@ -270,6 +285,11 @@ bool Generator::Generate(const FileDescriptor* file, // since they need to call static RegisterExtension() methods on these // classes. FixForeignFieldsInExtensions(); + // Descriptor options may have custom extensions. These custom options + // can only be successfully parsed after we register corresponding + // extensions. Therefore we parse all options again here to recognize + // custom options that may be unknown when we define the descriptors. + FixAllDescriptorOptions(); if (HasGenericServices(file)) { PrintServices(); } @@ -288,6 +308,13 @@ void Generator::PrintImports() const { module_name); } printer_->Print("\n"); + + // Print public imports. + for (int i = 0; i < file_->public_dependency_count(); ++i) { + string module_name = ModuleName(file_->public_dependency(i)->name()); + printer_->Print("from $module$ import *\n", "module", module_name); + } + printer_->Print("\n"); } // Prints the single file descriptor for this file. @@ -297,7 +324,7 @@ void Generator::PrintFileDescriptor() const { m["name"] = file_->name(); m["package"] = file_->package(); const char file_descriptor_template[] = - "$descriptor_name$ = descriptor.FileDescriptor(\n" + "$descriptor_name$ = _descriptor.FileDescriptor(\n" " name='$name$',\n" " package='$package$',\n"; printer_->Print(m, file_descriptor_template); @@ -321,6 +348,11 @@ void Generator::PrintTopLevelEnums() const { for (int i = 0; i < file_->enum_type_count(); ++i) { const EnumDescriptor& enum_descriptor = *file_->enum_type(i); PrintEnum(enum_descriptor); + printer_->Print("$name$ = " + "enum_type_wrapper.EnumTypeWrapper($descriptor_name$)", + "name", enum_descriptor.name(), + "descriptor_name", + ModuleLevelDescriptorName(enum_descriptor)); printer_->Print("\n"); for (int j = 0; j < enum_descriptor.value_count(); ++j) { @@ -355,7 +387,7 @@ void Generator::PrintEnum(const EnumDescriptor& enum_descriptor) const { m["full_name"] = enum_descriptor.full_name(); m["file"] = kDescriptorKey; const char enum_descriptor_template[] = - "$descriptor_name$ = descriptor.EnumDescriptor(\n" + "$descriptor_name$ = _descriptor.EnumDescriptor(\n" " name='$name$',\n" " full_name='$full_name$',\n" " filename=None,\n" @@ -436,7 +468,7 @@ void Generator::PrintServiceDescriptor( descriptor.options().SerializeToString(&options_string); printer_->Print( - "$service_name$ = descriptor.ServiceDescriptor(\n", + "$service_name$ = _descriptor.ServiceDescriptor(\n", "service_name", service_name); printer_->Indent(); map<string, string> m; @@ -459,7 +491,6 @@ void Generator::PrintServiceDescriptor( printer_->Print("methods=[\n"); for (int i = 0; i < descriptor.method_count(); ++i) { const MethodDescriptor* method = descriptor.method(i); - string options_string; method->options().SerializeToString(&options_string); m.clear(); @@ -470,7 +501,7 @@ void Generator::PrintServiceDescriptor( m["input_type"] = ModuleLevelDescriptorName(*(method->input_type())); m["output_type"] = ModuleLevelDescriptorName(*(method->output_type())); m["options_value"] = OptionsValue("MethodOptions", options_string); - printer_->Print("descriptor.MethodDescriptor(\n"); + printer_->Print("_descriptor.MethodDescriptor(\n"); printer_->Indent(); printer_->Print( m, @@ -491,7 +522,7 @@ void Generator::PrintServiceDescriptor( void Generator::PrintServiceClass(const ServiceDescriptor& descriptor) const { // Print the service. - printer_->Print("class $class_name$(service.Service):\n", + printer_->Print("class $class_name$(_service.Service):\n", "class_name", descriptor.name()); printer_->Indent(); printer_->Print( @@ -523,7 +554,7 @@ void Generator::PrintDescriptor(const Descriptor& message_descriptor) const { PrintNestedDescriptors(message_descriptor); printer_->Print("\n"); - printer_->Print("$descriptor_name$ = descriptor.Descriptor(\n", + printer_->Print("$descriptor_name$ = _descriptor.Descriptor(\n", "descriptor_name", ModuleLevelDescriptorName(message_descriptor)); printer_->Indent(); @@ -618,10 +649,10 @@ void Generator::PrintMessages() const { // Mutually recursive with PrintNestedMessages(). void Generator::PrintMessage( const Descriptor& message_descriptor) const { - printer_->Print("class $name$(message.Message):\n", "name", + printer_->Print("class $name$(_message.Message):\n", "name", message_descriptor.name()); printer_->Indent(); - printer_->Print("__metaclass__ = reflection.GeneratedProtocolMessageType\n"); + printer_->Print("__metaclass__ = _reflection.GeneratedProtocolMessageType\n"); PrintNestedMessages(message_descriptor); map<string, string> m; m["descriptor_key"] = kDescriptorKey; @@ -779,6 +810,7 @@ void Generator::FixForeignFieldsInExtensions() const { for (int i = 0; i < file_->message_type_count(); ++i) { FixForeignFieldsInNestedExtensions(*file_->message_type(i)); } + printer_->Print("\n"); } void Generator::FixForeignFieldsInExtension( @@ -829,7 +861,7 @@ void Generator::PrintEnumValueDescriptor( m["options"] = OptionsValue("EnumValueOptions", options_string); printer_->Print( m, - "descriptor.EnumValueDescriptor(\n" + "_descriptor.EnumValueDescriptor(\n" " name='$name$', index=$index$, number=$number$,\n" " options=$options$,\n" " type=None)"); @@ -843,7 +875,7 @@ string Generator::OptionsValue( return "None"; } else { string full_class_name = "descriptor_pb2." + class_name; - return "descriptor._ParseOptions(" + full_class_name + "(), '" + return "_descriptor._ParseOptions(" + full_class_name + "(), '" + CEscape(serialized_options)+ "')"; } } @@ -869,7 +901,7 @@ void Generator::PrintFieldDescriptor( // these fields in correctly after all referenced descriptors have been // defined and/or imported (see FixForeignFieldsInDescriptors()). const char field_descriptor_decl[] = - "descriptor.FieldDescriptor(\n" + "_descriptor.FieldDescriptor(\n" " name='$name$', full_name='$full_name$', index=$index$,\n" " number=$number$, type=$type$, cpp_type=$cpp_type$, label=$label$,\n" " has_default_value=$has_default_value$, default_value=$default_value$,\n" @@ -1000,6 +1032,125 @@ void Generator::PrintSerializedPbInterval( "serialized_end", SimpleItoa(offset + sp.size())); } +namespace { +void PrintDescriptorOptionsFixingCode(const string& descriptor, + const string& options, + io::Printer* printer) { + // TODO(xiaofeng): I have added a method _SetOptions() to DescriptorBase + // in proto2 python runtime but it couldn't be used here because appengine + // uses a snapshot version of the library in which the new method is not + // yet present. After appengine has synced their runtime library, the code + // below should be cleaned up to use _SetOptions(). + printer->Print( + "$descriptor$.has_options = True\n" + "$descriptor$._options = $options$\n", + "descriptor", descriptor, "options", options); +} +} // namespace + +// Prints expressions that set the options field of all descriptors. +void Generator::FixAllDescriptorOptions() const { + // Prints an expression that sets the file descriptor's options. + string file_options = OptionsValue( + "FileOptions", file_->options().SerializeAsString()); + if (file_options != "None") { + PrintDescriptorOptionsFixingCode(kDescriptorKey, file_options, printer_); + } + // Prints expressions that set the options for all top level enums. + for (int i = 0; i < file_->enum_type_count(); ++i) { + const EnumDescriptor& enum_descriptor = *file_->enum_type(i); + FixOptionsForEnum(enum_descriptor); + } + // Prints expressions that set the options for all top level extensions. + for (int i = 0; i < file_->extension_count(); ++i) { + const FieldDescriptor& field = *file_->extension(i); + FixOptionsForField(field); + } + // Prints expressions that set the options for all messages, nested enums, + // nested extensions and message fields. + for (int i = 0; i < file_->message_type_count(); ++i) { + FixOptionsForMessage(*file_->message_type(i)); + } +} + +// Prints expressions that set the options for an enum descriptor and its +// value descriptors. +void Generator::FixOptionsForEnum(const EnumDescriptor& enum_descriptor) const { + string descriptor_name = ModuleLevelDescriptorName(enum_descriptor); + string enum_options = OptionsValue( + "EnumOptions", enum_descriptor.options().SerializeAsString()); + if (enum_options != "None") { + PrintDescriptorOptionsFixingCode(descriptor_name, enum_options, printer_); + } + for (int i = 0; i < enum_descriptor.value_count(); ++i) { + const EnumValueDescriptor& value_descriptor = *enum_descriptor.value(i); + string value_options = OptionsValue( + "EnumValueOptions", value_descriptor.options().SerializeAsString()); + if (value_options != "None") { + PrintDescriptorOptionsFixingCode( + StringPrintf("%s.values_by_name[\"%s\"]", descriptor_name.c_str(), + value_descriptor.name().c_str()), + value_options, printer_); + } + } +} + +// Prints expressions that set the options for field descriptors (including +// extensions). +void Generator::FixOptionsForField( + const FieldDescriptor& field) const { + string field_options = OptionsValue( + "FieldOptions", field.options().SerializeAsString()); + if (field_options != "None") { + string field_name; + if (field.is_extension()) { + if (field.extension_scope() == NULL) { + // Top level extensions. + field_name = field.name(); + } else { + field_name = FieldReferencingExpression( + field.extension_scope(), field, "extensions_by_name"); + } + } else { + field_name = FieldReferencingExpression( + field.containing_type(), field, "fields_by_name"); + } + PrintDescriptorOptionsFixingCode(field_name, field_options, printer_); + } +} + +// Prints expressions that set the options for a message and all its inner +// types (nested messages, nested enums, extensions, fields). +void Generator::FixOptionsForMessage(const Descriptor& descriptor) const { + // Nested messages. + for (int i = 0; i < descriptor.nested_type_count(); ++i) { + FixOptionsForMessage(*descriptor.nested_type(i)); + } + // Enums. + for (int i = 0; i < descriptor.enum_type_count(); ++i) { + FixOptionsForEnum(*descriptor.enum_type(i)); + } + // Fields. + for (int i = 0; i < descriptor.field_count(); ++i) { + const FieldDescriptor& field = *descriptor.field(i); + FixOptionsForField(field); + } + // Extensions. + for (int i = 0; i < descriptor.extension_count(); ++i) { + const FieldDescriptor& field = *descriptor.extension(i); + FixOptionsForField(field); + } + // Message option for this message. + string message_options = OptionsValue( + "MessageOptions", descriptor.options().SerializeAsString()); + if (message_options != "None") { + string descriptor_name = ModuleLevelDescriptorName(descriptor); + PrintDescriptorOptionsFixingCode(descriptor_name, + message_options, + printer_); + } +} + } // namespace python } // namespace compiler } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.h b/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.h index 84eaf8a..a3f22ce 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.h +++ b/third_party/protobuf/src/google/protobuf/compiler/python/python_generator.h @@ -138,6 +138,11 @@ class LIBPROTOC_EXPORT Generator : public CodeGenerator { void PrintSerializedPbInterval( const DescriptorT& descriptor, DescriptorProtoT& proto) const; + void FixAllDescriptorOptions() const; + void FixOptionsForField(const FieldDescriptor& field) const; + void FixOptionsForEnum(const EnumDescriptor& descriptor) const; + void FixOptionsForMessage(const Descriptor& descriptor) const; + // Very coarse-grained lock to ensure that Generate() is reentrant. // Guards file_, printer_ and file_descriptor_serialized_. mutable Mutex mutex_; diff --git a/third_party/protobuf/src/google/protobuf/compiler/subprocess.cc b/third_party/protobuf/src/google/protobuf/compiler/subprocess.cc index 5fb5d5c..860fc87 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/subprocess.cc +++ b/third_party/protobuf/src/google/protobuf/compiler/subprocess.cc @@ -33,6 +33,7 @@ #include <google/protobuf/compiler/subprocess.h> #include <algorithm> +#include <iostream> #ifndef _WIN32 #include <errno.h> diff --git a/third_party/protobuf/src/google/protobuf/compiler/subprocess.h b/third_party/protobuf/src/google/protobuf/compiler/subprocess.h index 0056496..de9fce9 100644 --- a/third_party/protobuf/src/google/protobuf/compiler/subprocess.h +++ b/third_party/protobuf/src/google/protobuf/compiler/subprocess.h @@ -53,7 +53,7 @@ class Message; namespace compiler { // Utility class for launching sub-processes. -class LIBPROTOC_EXPORT Subprocess { +class Subprocess { public: Subprocess(); ~Subprocess(); diff --git a/third_party/protobuf/src/google/protobuf/descriptor.cc b/third_party/protobuf/src/google/protobuf/descriptor.cc index 5bdd565..45293fe 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor.cc +++ b/third_party/protobuf/src/google/protobuf/descriptor.cc @@ -54,7 +54,7 @@ #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> #include <google/protobuf/stubs/map-util.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> #undef PACKAGE // autoheader #defines this. :( @@ -108,6 +108,21 @@ const char * const FieldDescriptor::kTypeToName[MAX_TYPE + 1] = { "sint64", // TYPE_SINT64 }; +const char * const FieldDescriptor::kCppTypeToName[MAX_CPPTYPE + 1] = { + "ERROR", // 0 is reserved for errors + + "int32", // CPPTYPE_INT32 + "int64", // CPPTYPE_INT64 + "uint32", // CPPTYPE_UINT32 + "uint64", // CPPTYPE_UINT64 + "double", // CPPTYPE_DOUBLE + "float", // CPPTYPE_FLOAT + "bool", // CPPTYPE_BOOL + "enum", // CPPTYPE_ENUM + "string", // CPPTYPE_STRING + "message", // CPPTYPE_MESSAGE +}; + const char * const FieldDescriptor::kLabelToName[MAX_LABEL + 1] = { "ERROR", // 0 is reserved for errors @@ -221,7 +236,8 @@ struct PointerStringPairHash { struct Symbol { enum Type { - NULL_SYMBOL, MESSAGE, FIELD, ENUM, ENUM_VALUE, SERVICE, METHOD, PACKAGE + NULL_SYMBOL, MESSAGE, FIELD, ENUM, ENUM_VALUE, SERVICE, METHOD, + PACKAGE }; Type type; union { @@ -311,17 +327,42 @@ class DescriptorPool::Tables { Tables(); ~Tables(); - // Checkpoint the state of the tables. Future calls to Rollback() will - // return the Tables to this state. This is used when building files, since - // some kinds of validation errors cannot be detected until the file's - // descriptors have already been added to the tables. BuildFile() calls - // Checkpoint() before it starts building and Rollback() if it encounters - // an error. - void Checkpoint(); + // Record the current state of the tables to the stack of checkpoints. + // Each call to AddCheckpoint() must be paired with exactly one call to either + // ClearLastCheckpoint() or RollbackToLastCheckpoint(). + // + // This is used when building files, since some kinds of validation errors + // cannot be detected until the file's descriptors have already been added to + // the tables. + // + // This supports recursive checkpoints, since building a file may trigger + // recursive building of other files. Note that recursive checkpoints are not + // normally necessary; explicit dependencies are built prior to checkpointing. + // So although we recursively build transitive imports, there is at most one + // checkpoint in the stack during dependency building. + // + // Recursive checkpoints only arise during cross-linking of the descriptors. + // Symbol references must be resolved, via DescriptorBuilder::FindSymbol and + // friends. If the pending file references an unknown symbol + // (e.g., it is not defined in the pending file's explicit dependencies), and + // the pool is using a fallback database, and that database contains a file + // defining that symbol, and that file has not yet been built by the pool, + // the pool builds the file during cross-linking, leading to another + // checkpoint. + void AddCheckpoint(); + + // Mark the last checkpoint as having cleared successfully, removing it from + // the stack. If the stack is empty, all pending symbols will be committed. + // + // Note that this does not guarantee that the symbols added since the last + // checkpoint won't be rolled back: if a checkpoint gets rolled back, + // everything past that point gets rolled back, including symbols added after + // checkpoints that were pushed onto the stack after it and marked as cleared. + void ClearLastCheckpoint(); - // Roll back the Tables to the state of the last Checkpoint(), removing - // everything that was added after that point. - void Rollback(); + // Roll back the Tables to the state of the checkpoint at the top of the + // stack, removing everything that was added after that point. + void RollbackToLastCheckpoint(); // The stack of files which are currently being built. Used to detect // cyclic dependencies when loading files from a DescriptorDatabase. Not @@ -405,10 +446,28 @@ class DescriptorPool::Tables { FilesByNameMap files_by_name_; ExtensionsGroupedByDescriptorMap extensions_; - int strings_before_checkpoint_; - int messages_before_checkpoint_; - int file_tables_before_checkpoint_; - int allocations_before_checkpoint_; + struct CheckPoint { + explicit CheckPoint(const Tables* tables) + : strings_before_checkpoint(tables->strings_.size()), + messages_before_checkpoint(tables->messages_.size()), + file_tables_before_checkpoint(tables->file_tables_.size()), + allocations_before_checkpoint(tables->allocations_.size()), + pending_symbols_before_checkpoint( + tables->symbols_after_checkpoint_.size()), + pending_files_before_checkpoint( + tables->files_after_checkpoint_.size()), + pending_extensions_before_checkpoint( + tables->extensions_after_checkpoint_.size()) { + } + int strings_before_checkpoint; + int messages_before_checkpoint; + int file_tables_before_checkpoint; + int allocations_before_checkpoint; + int pending_symbols_before_checkpoint; + int pending_files_before_checkpoint; + int pending_extensions_before_checkpoint; + }; + vector<CheckPoint> checkpoints_; vector<const char* > symbols_after_checkpoint_; vector<const char* > files_after_checkpoint_; vector<DescriptorIntPair> extensions_after_checkpoint_; @@ -481,11 +540,15 @@ class FileDescriptorTables { }; DescriptorPool::Tables::Tables() - : strings_before_checkpoint_(0), - messages_before_checkpoint_(0), - allocations_before_checkpoint_(0) {} + // Start some hash_map and hash_set objects with a small # of buckets + : known_bad_files_(3), + extensions_loaded_from_db_(3), + symbols_by_name_(3), + files_by_name_(3) {} + DescriptorPool::Tables::~Tables() { + GOOGLE_DCHECK(checkpoints_.empty()); // Note that the deletion order is important, since the destructors of some // messages may refer to objects in allocations_. STLDeleteElements(&messages_); @@ -496,51 +559,80 @@ DescriptorPool::Tables::~Tables() { STLDeleteElements(&file_tables_); } -FileDescriptorTables::FileDescriptorTables() {} +FileDescriptorTables::FileDescriptorTables() + // Initialize all the hash tables to start out with a small # of buckets + : symbols_by_parent_(3), + fields_by_lowercase_name_(3), + fields_by_camelcase_name_(3), + fields_by_number_(3), + enum_values_by_number_(3) { +} + FileDescriptorTables::~FileDescriptorTables() {} const FileDescriptorTables FileDescriptorTables::kEmpty; -void DescriptorPool::Tables::Checkpoint() { - strings_before_checkpoint_ = strings_.size(); - messages_before_checkpoint_ = messages_.size(); - file_tables_before_checkpoint_ = file_tables_.size(); - allocations_before_checkpoint_ = allocations_.size(); +void DescriptorPool::Tables::AddCheckpoint() { + checkpoints_.push_back(CheckPoint(this)); +} - symbols_after_checkpoint_.clear(); - files_after_checkpoint_.clear(); - extensions_after_checkpoint_.clear(); +void DescriptorPool::Tables::ClearLastCheckpoint() { + GOOGLE_DCHECK(!checkpoints_.empty()); + checkpoints_.pop_back(); + if (checkpoints_.empty()) { + // All checkpoints have been cleared: we can now commit all of the pending + // data. + symbols_after_checkpoint_.clear(); + files_after_checkpoint_.clear(); + extensions_after_checkpoint_.clear(); + } } -void DescriptorPool::Tables::Rollback() { - for (int i = 0; i < symbols_after_checkpoint_.size(); i++) { +void DescriptorPool::Tables::RollbackToLastCheckpoint() { + GOOGLE_DCHECK(!checkpoints_.empty()); + const CheckPoint& checkpoint = checkpoints_.back(); + + for (int i = checkpoint.pending_symbols_before_checkpoint; + i < symbols_after_checkpoint_.size(); + i++) { symbols_by_name_.erase(symbols_after_checkpoint_[i]); } - for (int i = 0; i < files_after_checkpoint_.size(); i++) { + for (int i = checkpoint.pending_files_before_checkpoint; + i < files_after_checkpoint_.size(); + i++) { files_by_name_.erase(files_after_checkpoint_[i]); } - for (int i = 0; i < extensions_after_checkpoint_.size(); i++) { + for (int i = checkpoint.pending_extensions_before_checkpoint; + i < extensions_after_checkpoint_.size(); + i++) { extensions_.erase(extensions_after_checkpoint_[i]); } - symbols_after_checkpoint_.clear(); - files_after_checkpoint_.clear(); - extensions_after_checkpoint_.clear(); + symbols_after_checkpoint_.resize( + checkpoint.pending_symbols_before_checkpoint); + files_after_checkpoint_.resize(checkpoint.pending_files_before_checkpoint); + extensions_after_checkpoint_.resize( + checkpoint.pending_extensions_before_checkpoint); STLDeleteContainerPointers( - strings_.begin() + strings_before_checkpoint_, strings_.end()); + strings_.begin() + checkpoint.strings_before_checkpoint, strings_.end()); STLDeleteContainerPointers( - messages_.begin() + messages_before_checkpoint_, messages_.end()); + messages_.begin() + checkpoint.messages_before_checkpoint, + messages_.end()); STLDeleteContainerPointers( - file_tables_.begin() + file_tables_before_checkpoint_, file_tables_.end()); - for (int i = allocations_before_checkpoint_; i < allocations_.size(); i++) { + file_tables_.begin() + checkpoint.file_tables_before_checkpoint, + file_tables_.end()); + for (int i = checkpoint.allocations_before_checkpoint; + i < allocations_.size(); + i++) { operator delete(allocations_[i]); } - strings_.resize(strings_before_checkpoint_); - messages_.resize(messages_before_checkpoint_); - file_tables_.resize(file_tables_before_checkpoint_); - allocations_.resize(allocations_before_checkpoint_); + strings_.resize(checkpoint.strings_before_checkpoint); + messages_.resize(checkpoint.messages_before_checkpoint); + file_tables_.resize(checkpoint.file_tables_before_checkpoint); + allocations_.resize(checkpoint.allocations_before_checkpoint); + checkpoints_.pop_back(); } // ------------------------------------------------------------------- @@ -811,7 +903,7 @@ void DeleteGeneratedPool() { generated_pool_ = NULL; } -void InitGeneratedPool() { +static void InitGeneratedPool() { generated_database_ = new EncodedDescriptorDatabase; generated_pool_ = new DescriptorPool(generated_database_); @@ -874,11 +966,11 @@ const FileDescriptor* DescriptorPool::FindFileByName(const string& name) const { const FileDescriptor* result = tables_->FindFile(name); if (result != NULL) return result; if (underlay_ != NULL) { - const FileDescriptor* result = underlay_->FindFileByName(name); + result = underlay_->FindFileByName(name); if (result != NULL) return result; } if (TryFindFileInFallbackDatabase(name)) { - const FileDescriptor* result = tables_->FindFile(name); + result = tables_->FindFile(name); if (result != NULL) return result; } return NULL; @@ -890,12 +982,12 @@ const FileDescriptor* DescriptorPool::FindFileContainingSymbol( Symbol result = tables_->FindSymbol(symbol_name); if (!result.IsNull()) return result.GetFile(); if (underlay_ != NULL) { - const FileDescriptor* result = + const FileDescriptor* file_result = underlay_->FindFileContainingSymbol(symbol_name); - if (result != NULL) return result; + if (file_result != NULL) return file_result; } if (TryFindSymbolInFallbackDatabase(symbol_name)) { - Symbol result = tables_->FindSymbol(symbol_name); + result = tables_->FindSymbol(symbol_name); if (!result.IsNull()) return result.GetFile(); } return NULL; @@ -962,12 +1054,11 @@ const FieldDescriptor* DescriptorPool::FindExtensionByNumber( return result; } if (underlay_ != NULL) { - const FieldDescriptor* result = - underlay_->FindExtensionByNumber(extendee, number); + result = underlay_->FindExtensionByNumber(extendee, number); if (result != NULL) return result; } if (TryFindExtensionInFallbackDatabase(extendee, number)) { - const FieldDescriptor* result = tables_->FindExtension(extendee, number); + result = tables_->FindExtension(extendee, number); if (result != NULL) { return result; } @@ -1241,9 +1332,48 @@ bool DescriptorPool::TryFindFileInFallbackDatabase(const string& name) const { return true; } +bool DescriptorPool::IsSubSymbolOfBuiltType(const string& name) const { + string prefix = name; + for (;;) { + string::size_type dot_pos = prefix.find_last_of('.'); + if (dot_pos == string::npos) { + break; + } + prefix = prefix.substr(0, dot_pos); + Symbol symbol = tables_->FindSymbol(prefix); + // If the symbol type is anything other than PACKAGE, then its complete + // definition is already known. + if (!symbol.IsNull() && symbol.type != Symbol::PACKAGE) { + return true; + } + } + if (underlay_ != NULL) { + // Check to see if any prefix of this symbol exists in the underlay. + return underlay_->IsSubSymbolOfBuiltType(name); + } + return false; +} + bool DescriptorPool::TryFindSymbolInFallbackDatabase(const string& name) const { if (fallback_database_ == NULL) return false; + // We skip looking in the fallback database if the name is a sub-symbol of + // any descriptor that already exists in the descriptor pool (except for + // package descriptors). This is valid because all symbols except for + // packages are defined in a single file, so if the symbol exists then we + // should already have its definition. + // + // The other reason to do this is to support "overriding" type definitions + // by merging two databases that define the same type. (Yes, people do + // this.) The main difficulty with making this work is that + // FindFileContainingSymbol() is allowed to return both false positives + // (e.g., SimpleDescriptorDatabase, UpgradedDescriptorDatabase) and false + // negatives (e.g. ProtoFileParser, SourceTreeDescriptorDatabase). When two + // such databases are merged, looking up a non-existent sub-symbol of a type + // that already exists in the descriptor pool can result in an attempt to + // load multiple definitions of the same type. The check below avoids this. + if (IsSubSymbolOfBuiltType(name)) return false; + FileDescriptorProto file_proto; if (!fallback_database_->FindFileContainingSymbol(name, &file_proto)) { return false; @@ -1345,6 +1475,14 @@ void FileDescriptor::CopyTo(FileDescriptorProto* proto) const { proto->add_dependency(dependency(i)->name()); } + for (int i = 0; i < public_dependency_count(); i++) { + proto->add_public_dependency(public_dependencies_[i]); + } + + for (int i = 0; i < weak_dependency_count(); i++) { + proto->add_weak_dependency(weak_dependencies_[i]); + } + for (int i = 0; i < message_type_count(); i++) { message_type(i)->CopyTo(proto->add_message_type()); } @@ -1363,6 +1501,12 @@ void FileDescriptor::CopyTo(FileDescriptorProto* proto) const { } } +void FileDescriptor::CopySourceCodeInfoTo(FileDescriptorProto* proto) const { + if (source_code_info_ != &SourceCodeInfo::default_instance()) { + proto->mutable_source_code_info()->CopyFrom(*source_code_info_); + } +} + void Descriptor::CopyTo(DescriptorProto* proto) const { proto->set_name(name()); @@ -1490,16 +1634,14 @@ void MethodDescriptor::CopyTo(MethodDescriptorProto* proto) const { namespace { // Used by each of the option formatters. -bool RetrieveOptions(const Message &options, vector<string> *option_entries) { +bool RetrieveOptions(int depth, + const Message &options, + vector<string> *option_entries) { option_entries->clear(); const Reflection* reflection = options.GetReflection(); vector<const FieldDescriptor*> fields; reflection->ListFields(options, &fields); for (int i = 0; i < fields.size(); i++) { - // Doesn't make sense to have message type fields here - if (fields[i]->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { - continue; - } int count = 1; bool repeated = false; if (fields[i]->is_repeated()) { @@ -1508,9 +1650,27 @@ bool RetrieveOptions(const Message &options, vector<string> *option_entries) { } for (int j = 0; j < count; j++) { string fieldval; - TextFormat::PrintFieldValueToString(options, fields[i], - repeated ? count : -1, &fieldval); - option_entries->push_back(fields[i]->name() + " = " + fieldval); + if (fields[i]->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { + string tmp; + TextFormat::Printer printer; + printer.SetInitialIndentLevel(depth + 1); + printer.PrintFieldValueToString(options, fields[i], + repeated ? j : -1, &tmp); + fieldval.append("{\n"); + fieldval.append(tmp); + fieldval.append(depth * 2, ' '); + fieldval.append("}"); + } else { + TextFormat::PrintFieldValueToString(options, fields[i], + repeated ? j : -1, &fieldval); + } + string name; + if (fields[i]->is_extension()) { + name = "(." + fields[i]->full_name() + ")"; + } else { + name = fields[i]->name(); + } + option_entries->push_back(name + " = " + fieldval); } } return !option_entries->empty(); @@ -1518,9 +1678,9 @@ bool RetrieveOptions(const Message &options, vector<string> *option_entries) { // Formats options that all appear together in brackets. Does not include // brackets. -bool FormatBracketedOptions(const Message &options, string *output) { +bool FormatBracketedOptions(int depth, const Message &options, string *output) { vector<string> all_options; - if (RetrieveOptions(options, &all_options)) { + if (RetrieveOptions(depth, options, &all_options)) { output->append(JoinStrings(all_options, ", ")); } return !all_options.empty(); @@ -1530,7 +1690,7 @@ bool FormatBracketedOptions(const Message &options, string *output) { bool FormatLineOptions(int depth, const Message &options, string *output) { string prefix(depth * 2, ' '); vector<string> all_options; - if (RetrieveOptions(options, &all_options)) { + if (RetrieveOptions(depth, options, &all_options)) { for (int i = 0; i < all_options.size(); i++) { strings::SubstituteAndAppend(output, "$0option $1;\n", prefix, all_options[i]); @@ -1544,9 +1704,24 @@ bool FormatLineOptions(int depth, const Message &options, string *output) { string FileDescriptor::DebugString() const { string contents = "syntax = \"proto2\";\n\n"; + set<int> public_dependencies; + set<int> weak_dependencies; + public_dependencies.insert(public_dependencies_, + public_dependencies_ + public_dependency_count_); + weak_dependencies.insert(weak_dependencies_, + weak_dependencies_ + weak_dependency_count_); + for (int i = 0; i < dependency_count(); i++) { - strings::SubstituteAndAppend(&contents, "import \"$0\";\n", - dependency(i)->name()); + if (public_dependencies.count(i) > 0) { + strings::SubstituteAndAppend(&contents, "import public \"$0\";\n", + dependency(i)->name()); + } else if (weak_dependencies.count(i) > 0) { + strings::SubstituteAndAppend(&contents, "import weak \"$0\";\n", + dependency(i)->name()); + } else { + strings::SubstituteAndAppend(&contents, "import \"$0\";\n", + dependency(i)->name()); + } } if (!package().empty()) { @@ -1712,7 +1887,7 @@ void FieldDescriptor::DebugString(int depth, string *contents) const { } string formatted_options; - if (FormatBracketedOptions(options(), &formatted_options)) { + if (FormatBracketedOptions(depth, options(), &formatted_options)) { contents->append(bracketed ? ", " : " ["); bracketed = true; contents->append(formatted_options); @@ -1761,7 +1936,7 @@ void EnumValueDescriptor::DebugString(int depth, string *contents) const { prefix, name(), number()); string formatted_options; - if (FormatBracketedOptions(options(), &formatted_options)) { + if (FormatBracketedOptions(depth, options(), &formatted_options)) { strings::SubstituteAndAppend(contents, " [$0]", formatted_options); } contents->append(";\n"); @@ -1807,6 +1982,126 @@ void MethodDescriptor::DebugString(int depth, string *contents) const { contents->append(";\n"); } } + + +// Location methods =============================================== + +static bool PathsEqual(const vector<int>& x, const RepeatedField<int32>& y) { + if (x.size() != y.size()) return false; + for (int i = 0; i < x.size(); ++i) { + if (x[i] != y.Get(i)) return false; + } + return true; +} + +bool FileDescriptor::GetSourceLocation(const vector<int>& path, + SourceLocation* out_location) const { + GOOGLE_CHECK_NOTNULL(out_location); + const SourceCodeInfo* info = source_code_info_; + for (int i = 0; info && i < info->location_size(); ++i) { + if (PathsEqual(path, info->location(i).path())) { + const RepeatedField<int32>& span = info->location(i).span(); + if (span.size() == 3 || span.size() == 4) { + out_location->start_line = span.Get(0); + out_location->start_column = span.Get(1); + out_location->end_line = span.Get(span.size() == 3 ? 0 : 2); + out_location->end_column = span.Get(span.size() - 1); + + out_location->leading_comments = info->location(i).leading_comments(); + out_location->trailing_comments = info->location(i).trailing_comments(); + return true; + } + } + } + return false; +} + +bool FieldDescriptor::is_packed() const { + return is_packable() && (options_ != NULL) && options_->packed(); +} + +bool Descriptor::GetSourceLocation(SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return file()->GetSourceLocation(path, out_location); +} + +bool FieldDescriptor::GetSourceLocation(SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return file()->GetSourceLocation(path, out_location); +} + +bool EnumDescriptor::GetSourceLocation(SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return file()->GetSourceLocation(path, out_location); +} + +bool MethodDescriptor::GetSourceLocation(SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return service()->file()->GetSourceLocation(path, out_location); +} + +bool ServiceDescriptor::GetSourceLocation(SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return file()->GetSourceLocation(path, out_location); +} + +bool EnumValueDescriptor::GetSourceLocation( + SourceLocation* out_location) const { + vector<int> path; + GetLocationPath(&path); + return type()->file()->GetSourceLocation(path, out_location); +} + +void Descriptor::GetLocationPath(vector<int>* output) const { + if (containing_type()) { + containing_type()->GetLocationPath(output); + output->push_back(DescriptorProto::kNestedTypeFieldNumber); + output->push_back(index()); + } else { + output->push_back(FileDescriptorProto::kMessageTypeFieldNumber); + output->push_back(index()); + } +} + +void FieldDescriptor::GetLocationPath(vector<int>* output) const { + containing_type()->GetLocationPath(output); + output->push_back(DescriptorProto::kFieldFieldNumber); + output->push_back(index()); +} + +void EnumDescriptor::GetLocationPath(vector<int>* output) const { + if (containing_type()) { + containing_type()->GetLocationPath(output); + output->push_back(DescriptorProto::kEnumTypeFieldNumber); + output->push_back(index()); + } else { + output->push_back(FileDescriptorProto::kEnumTypeFieldNumber); + output->push_back(index()); + } +} + +void EnumValueDescriptor::GetLocationPath(vector<int>* output) const { + type()->GetLocationPath(output); + output->push_back(EnumDescriptorProto::kValueFieldNumber); + output->push_back(index()); +} + +void ServiceDescriptor::GetLocationPath(vector<int>* output) const { + output->push_back(FileDescriptorProto::kServiceFieldNumber); + output->push_back(index()); +} + +void MethodDescriptor::GetLocationPath(vector<int>* output) const { + service()->GetLocationPath(output); + output->push_back(ServiceDescriptorProto::kMethodFieldNumber); + output->push_back(index()); +} + // =================================================================== namespace { @@ -1859,6 +2154,7 @@ class DescriptorBuilder { string filename_; FileDescriptor* file_; FileDescriptorTables* file_tables_; + set<const FileDescriptor*> dependencies_; // If LookupSymbol() finds a symbol that is in a file which is not a declared // dependency of this file, it will fail, but will set @@ -1888,6 +2184,10 @@ class DescriptorBuilder { // nested package within package_name. bool IsInPackage(const FileDescriptor* file, const string& package_name); + // Helper function which finds all public dependencies of the given file, and + // stores the them in the dependencies_ set in the builder. + void RecordPublicDependencies(const FileDescriptor* file); + // Like tables_->FindSymbol(), but additionally: // - Search the pool's underlay if not found in tables_. // - Insure that the resulting Symbol is from one of the file's declared @@ -1898,6 +2198,10 @@ class DescriptorBuilder { // file's declared dependencies. Symbol FindSymbolNotEnforcingDeps(const string& name); + // This implements the body of FindSymbolNotEnforcingDeps(). + Symbol FindSymbolNotEnforcingDepsHelper(const DescriptorPool* pool, + const string& name); + // Like FindSymbol(), but looks up the name relative to some other symbol // name. This first searches siblings of relative_to, then siblings of its // parents, etc. For example, LookupSymbol("foo.bar", "baz.qux.corge") makes @@ -2189,6 +2493,7 @@ class DescriptorBuilder { void ValidateMapKey(FieldDescriptor* field, const FieldDescriptorProto& proto); + }; const FileDescriptor* DescriptorPool::BuildFile( @@ -2275,31 +2580,48 @@ bool DescriptorBuilder::IsInPackage(const FileDescriptor* file, file->package()[package_name.size()] == '.'); } -Symbol DescriptorBuilder::FindSymbolNotEnforcingDeps(const string& name) { - Symbol result; +void DescriptorBuilder::RecordPublicDependencies(const FileDescriptor* file) { + if (file == NULL || !dependencies_.insert(file).second) return; + for (int i = 0; file != NULL && i < file->public_dependency_count(); i++) { + RecordPublicDependencies(file->public_dependency(i)); + } +} - // We need to search our pool and all its underlays. - const DescriptorPool* pool = pool_; - while (true) { - // If we are looking at an underlay, we must lock its mutex_, since we are - // accessing the underlay's tables_ dircetly. - MutexLockMaybe lock((pool == pool_) ? NULL : pool->mutex_); +Symbol DescriptorBuilder::FindSymbolNotEnforcingDepsHelper( + const DescriptorPool* pool, const string& name) { + // If we are looking at an underlay, we must lock its mutex_, since we are + // accessing the underlay's tables_ directly. + MutexLockMaybe lock((pool == pool_) ? NULL : pool->mutex_); + + Symbol result = pool->tables_->FindSymbol(name); + if (result.IsNull() && pool->underlay_ != NULL) { + // Symbol not found; check the underlay. + result = FindSymbolNotEnforcingDepsHelper(pool->underlay_, name); + } - // Note that we don't have to check fallback_database_ here because the - // symbol has to be in one of its file's direct dependencies, and we have - // already loaded those by the time we get here. - result = pool->tables_->FindSymbol(name); - if (!result.IsNull()) break; - if (pool->underlay_ == NULL) return kNullSymbol; - pool = pool->underlay_; + if (result.IsNull()) { + // In theory, we shouldn't need to check fallback_database_ because the + // symbol should be in one of its file's direct dependencies, and we have + // already loaded those by the time we get here. But we check anyway so + // that we can generate better error message when dependencies are missing + // (i.e., "missing dependency" rather than "type is not defined"). + if (pool->TryFindSymbolInFallbackDatabase(name)) { + result = pool->tables_->FindSymbol(name); + } } return result; } +Symbol DescriptorBuilder::FindSymbolNotEnforcingDeps(const string& name) { + return FindSymbolNotEnforcingDepsHelper(pool_, name); +} + Symbol DescriptorBuilder::FindSymbol(const string& name) { Symbol result = FindSymbolNotEnforcingDeps(name); + if (result.IsNull()) return result; + if (!pool_->enforce_dependencies_) { // Hack for CompilerUpgrader. return result; @@ -2308,10 +2630,7 @@ Symbol DescriptorBuilder::FindSymbol(const string& name) { // Only find symbols which were defined in this file or one of its // dependencies. const FileDescriptor* file = result.GetFile(); - if (file == file_) return result; - for (int i = 0; i < file_->dependency_count(); i++) { - if (file == file_->dependency(i)) return result; - } + if (file == file_ || dependencies_.count(file) > 0) return result; if (result.type == Symbol::PACKAGE) { // Arg, this is overcomplicated. The symbol is a package name. It could @@ -2322,12 +2641,10 @@ Symbol DescriptorBuilder::FindSymbol(const string& name) { // dependency also defines the same package. We can't really rule out this // symbol unless none of the dependencies define it. if (IsInPackage(file_, name)) return result; - for (int i = 0; i < file_->dependency_count(); i++) { + for (set<const FileDescriptor*>::const_iterator it = dependencies_.begin(); + it != dependencies_.end(); ++it) { // Note: A dependency may be NULL if it was not found or had errors. - if (file_->dependency(i) != NULL && - IsInPackage(file_->dependency(i), name)) { - return result; - } + if (*it != NULL && IsInPackage(*it, name)) return result; } } @@ -2448,6 +2765,8 @@ Symbol DescriptorBuilder::NewPlaceholder(const string& name, FileDescriptor* placeholder_file = tables_->Allocate<FileDescriptor>(); memset(placeholder_file, 0, sizeof(*placeholder_file)); + placeholder_file->source_code_info_ = &SourceCodeInfo::default_instance(); + placeholder_file->name_ = tables_->AllocateString(*placeholder_full_name + ".placeholder.proto"); placeholder_file->package_ = placeholder_package; @@ -2757,11 +3076,19 @@ const FileDescriptor* DescriptorBuilder::BuildFile( } // Checkpoint the tables so that we can roll back if something goes wrong. - tables_->Checkpoint(); + tables_->AddCheckpoint(); FileDescriptor* result = tables_->Allocate<FileDescriptor>(); file_ = result; + if (proto.has_source_code_info()) { + SourceCodeInfo *info = tables_->AllocateMessage<SourceCodeInfo>(); + info->CopyFrom(proto.source_code_info()); + result->source_code_info_ = info; + } else { + result->source_code_info_ = &SourceCodeInfo::default_instance(); + } + file_tables_ = tables_->AllocateFileTables(); file_->tables_ = file_tables_; @@ -2788,7 +3115,7 @@ const FileDescriptor* DescriptorBuilder::BuildFile( "A file with this name is already in the pool."); // Bail out early so that if this is actually the exact same file, we // don't end up reporting that every single symbol is already defined. - tables_->Rollback(); + tables_->RollbackToLastCheckpoint(); return NULL; } if (!result->package().empty()) { @@ -2833,6 +3160,45 @@ const FileDescriptor* DescriptorBuilder::BuildFile( result->dependencies_[i] = dependency; } + // Check public dependencies. + int public_dependency_count = 0; + result->public_dependencies_ = tables_->AllocateArray<int>( + proto.public_dependency_size()); + for (int i = 0; i < proto.public_dependency_size(); i++) { + // Only put valid public dependency indexes. + int index = proto.public_dependency(i); + if (index >= 0 && index < proto.dependency_size()) { + result->public_dependencies_[public_dependency_count++] = index; + } else { + AddError(proto.name(), proto, + DescriptorPool::ErrorCollector::OTHER, + "Invalid public dependency index."); + } + } + result->public_dependency_count_ = public_dependency_count; + + // Build dependency set + dependencies_.clear(); + for (int i = 0; i < result->dependency_count(); i++) { + RecordPublicDependencies(result->dependency(i)); + } + + // Check weak dependencies. + int weak_dependency_count = 0; + result->weak_dependencies_ = tables_->AllocateArray<int>( + proto.weak_dependency_size()); + for (int i = 0; i < proto.weak_dependency_size(); i++) { + int index = proto.weak_dependency(i); + if (index >= 0 && index < proto.dependency_size()) { + result->weak_dependencies_[weak_dependency_count++] = index; + } else { + AddError(proto.name(), proto, + DescriptorPool::ErrorCollector::OTHER, + "Invalid weak dependency index."); + } + } + result->weak_dependency_count_ = weak_dependency_count; + // Convert children. BUILD_ARRAY(proto, result, message_type, BuildMessage , NULL); BUILD_ARRAY(proto, result, enum_type , BuildEnum , NULL); @@ -2870,10 +3236,10 @@ const FileDescriptor* DescriptorBuilder::BuildFile( } if (had_errors_) { - tables_->Rollback(); + tables_->RollbackToLastCheckpoint(); return NULL; } else { - tables_->Checkpoint(); + tables_->ClearLastCheckpoint(); return result; } } @@ -3065,7 +3431,7 @@ void DescriptorBuilder::BuildFieldOrExtension(const FieldDescriptorProto& proto, UnescapeCEscapeString(proto.default_value())); } else { result->default_value_string_ = - tables_->AllocateString(proto.default_value()); + tables_->AllocateString(proto.default_value()); } break; case FieldDescriptor::CPPTYPE_MESSAGE: @@ -3126,7 +3492,15 @@ void DescriptorBuilder::BuildFieldOrExtension(const FieldDescriptorProto& proto, if (result->number() <= 0) { AddError(result->full_name(), proto, DescriptorPool::ErrorCollector::NUMBER, "Field numbers must be positive integers."); - } else if (result->number() > FieldDescriptor::kMaxNumber) { + } else if (!is_extension && result->number() > FieldDescriptor::kMaxNumber) { + // Only validate that the number is within the valid field range if it is + // not an extension. Since extension numbers are validated with the + // extendee's valid set of extension numbers, and those are in turn + // validated against the max allowed number, the check is unnecessary for + // extension fields. + // This avoids cross-linking issues that arise when attempting to check if + // the extendee is a message_set_wire_format message, which has a higher max + // on extension numbers. AddError(result->full_name(), proto, DescriptorPool::ErrorCollector::NUMBER, strings::Substitute("Field numbers cannot be greater than $0.", FieldDescriptor::kMaxNumber)); @@ -3181,12 +3555,10 @@ void DescriptorBuilder::BuildExtensionRange( "Extension numbers must be positive integers."); } - if (result->end > FieldDescriptor::kMaxNumber + 1) { - AddError(parent->full_name(), proto, - DescriptorPool::ErrorCollector::NUMBER, - strings::Substitute("Extension numbers cannot be greater than $0.", - FieldDescriptor::kMaxNumber)); - } + // Checking of the upper bound of the extension range is deferred until after + // options interpreting. This allows messages with message_set_wire_format to + // have extensions beyond FieldDescriptor::kMaxNumber, since the extension + // numbers are actually used as int32s in the message_set_wire_format. if (result->start >= result->end) { AddError(parent->full_name(), proto, @@ -3681,6 +4053,20 @@ void DescriptorBuilder::ValidateMessageOptions(Descriptor* message, VALIDATE_OPTIONS_FROM_ARRAY(message, nested_type, Message); VALIDATE_OPTIONS_FROM_ARRAY(message, enum_type, Enum); VALIDATE_OPTIONS_FROM_ARRAY(message, extension, Field); + + const int64 max_extension_range = + static_cast<int64>(message->options().message_set_wire_format() ? + kint32max : + FieldDescriptor::kMaxNumber); + for (int i = 0; i < message->extension_range_count(); ++i) { + if (message->extension_range(i)->end > max_extension_range + 1) { + AddError( + message->full_name(), proto.extension_range(i), + DescriptorPool::ErrorCollector::NUMBER, + strings::Substitute("Extension numbers cannot be greater than $0.", + max_extension_range)); + } + } } void DescriptorBuilder::ValidateFieldOptions(FieldDescriptor* field, @@ -3689,6 +4075,15 @@ void DescriptorBuilder::ValidateFieldOptions(FieldDescriptor* field, ValidateMapKey(field, proto); } + // Only message type fields may be lazy. + if (field->options().lazy()) { + if (field->type() != FieldDescriptor::TYPE_MESSAGE) { + AddError(field->full_name(), proto, + DescriptorPool::ErrorCollector::TYPE, + "[lazy = true] can only be specified for submessage fields."); + } + } + // Only repeated primitive fields may be packed. if (field->options().packed() && !field->is_packable()) { AddError( @@ -3727,11 +4122,27 @@ void DescriptorBuilder::ValidateFieldOptions(FieldDescriptor* field, "files. Note that you cannot extend a non-lite type to contain " "a lite type, but the reverse is allowed."); } + } void DescriptorBuilder::ValidateEnumOptions(EnumDescriptor* enm, const EnumDescriptorProto& proto) { VALIDATE_OPTIONS_FROM_ARRAY(enm, value, EnumValue); + if (!enm->options().allow_alias()) { + map<int, string> used_values; + for (int i = 0; i < enm->value_count(); ++i) { + const EnumValueDescriptor* enum_value = enm->value(i); + if (used_values.find(enum_value->number()) != used_values.end()) { + AddError(enm->full_name(), proto, + DescriptorPool::ErrorCollector::NUMBER, + "\"" + enum_value->full_name() + + "\" uses the same enum value as \"" + + used_values[enum_value->number()] + "\""); + } else { + used_values[enum_value->number()] = enum_value->full_name(); + } + } + } } void DescriptorBuilder::ValidateEnumValueOptions( @@ -3811,6 +4222,7 @@ void DescriptorBuilder::ValidateMapKey(FieldDescriptor* field, field->experimental_map_key_ = key_field; } + #undef VALIDATE_OPTIONS_FROM_ARRAY // ------------------------------------------------------------------- @@ -3901,9 +4313,11 @@ bool DescriptorBuilder::OptionInterpreter::InterpretSingleOption( // file we're currently building. The descriptor should be there as long as // the file we're building imported "google/protobuf/descriptors.proto". - // Note that we use DescriptorBuilder::FindSymbol(), not + // Note that we use DescriptorBuilder::FindSymbolNotEnforcingDeps(), not // DescriptorPool::FindMessageTypeByName() because we're already holding the - // pool's mutex, and the latter method locks it again. + // pool's mutex, and the latter method locks it again. We don't use + // FindSymbol() because files that use custom options only need to depend on + // the file that defines the option, not descriptor.proto itself. Symbol symbol = builder_->FindSymbolNotEnforcingDeps( options->GetDescriptor()->full_name()); if (!symbol.IsNull() && symbol.type == Symbol::MESSAGE) { @@ -3939,8 +4353,8 @@ bool DescriptorBuilder::OptionInterpreter::InterpretSingleOption( // DescriptorPool::FindExtensionByName(), for two reasons: 1) It allows // relative lookups, and 2) because we're already holding the pool's // mutex, and the latter method locks it again. - Symbol symbol = builder_->LookupSymbol(name_part, - options_to_interpret_->name_scope); + symbol = builder_->LookupSymbol(name_part, + options_to_interpret_->name_scope); if (!symbol.IsNull() && symbol.type == Symbol::FIELD) { field = symbol.field_descriptor; } @@ -4344,14 +4758,32 @@ class DescriptorBuilder::OptionInterpreter::AggregateOptionFinder virtual const FieldDescriptor* FindExtension( Message* message, const string& name) const { assert_mutex_held(builder_->pool_); + const Descriptor* descriptor = message->GetDescriptor(); Symbol result = builder_->LookupSymbolNoPlaceholder( - name, message->GetDescriptor()->full_name()); + name, descriptor->full_name()); if (result.type == Symbol::FIELD && result.field_descriptor->is_extension()) { return result.field_descriptor; - } else { - return NULL; + } else if (result.type == Symbol::MESSAGE && + descriptor->options().message_set_wire_format()) { + const Descriptor* foreign_type = result.descriptor; + // The text format allows MessageSet items to be specified using + // the type name, rather than the extension identifier. If the symbol + // lookup returned a Message, and the enclosing Message has + // message_set_wire_format = true, then return the message set + // extension, if one exists. + for (int i = 0; i < foreign_type->extension_count(); i++) { + const FieldDescriptor* extension = foreign_type->extension(i); + if (extension->containing_type() == descriptor && + extension->type() == FieldDescriptor::TYPE_MESSAGE && + extension->is_optional() && + extension->message_type() == foreign_type) { + // Found it. + return extension; + } + } } + return NULL; } }; @@ -4409,7 +4841,13 @@ bool DescriptorBuilder::OptionInterpreter::SetAggregateOption( } else { string serial; dynamic->SerializeToString(&serial); // Never fails - unknown_fields->AddLengthDelimited(option_field->number(), serial); + if (option_field->type() == FieldDescriptor::TYPE_MESSAGE) { + unknown_fields->AddLengthDelimited(option_field->number(), serial); + } else { + GOOGLE_CHECK_EQ(option_field->type(), FieldDescriptor::TYPE_GROUP); + UnknownFieldSet* group = unknown_fields->AddGroup(option_field->number()); + group->ParseFromString(serial); + } return true; } } diff --git a/third_party/protobuf/src/google/protobuf/descriptor.h b/third_party/protobuf/src/google/protobuf/descriptor.h index 7f87dd8..410d3ed 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor.h +++ b/third_party/protobuf/src/google/protobuf/descriptor.h @@ -89,6 +89,7 @@ class ServiceOptions; class MethodOptions; class FileOptions; class UninterpretedOption; +class SourceCodeInfo; // Defined in message.h class Message; @@ -100,6 +101,20 @@ class FileDescriptorTables; // Defined in unknown_field_set.h. class UnknownField; +// NB, all indices are zero-based. +struct SourceLocation { + int start_line; + int end_line; + int start_column; + int end_column; + + // Doc comments found at the source location. + // TODO(kenton): Maybe this struct should have been named SourceInfo or + // something instead. Oh well. + string leading_comments; + string trailing_comments; +}; + // Describes a type of protocol message, or a particular group within a // message. To obtain the Descriptor for a given message object, call // Message::GetDescriptor(). Generated message classes also have a @@ -236,6 +251,13 @@ class LIBPROTOBUF_EXPORT Descriptor { // this message type's scope. const FieldDescriptor* FindExtensionByCamelcaseName(const string& name) const; + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this message declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef MessageOptions OptionsType; @@ -243,6 +265,10 @@ class LIBPROTOBUF_EXPORT Descriptor { // correct depth void DebugString(int depth, string *contents) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; const FileDescriptor* file_; @@ -388,15 +414,19 @@ class LIBPROTOBUF_EXPORT FieldDescriptor { // when parsing formats which prefer to use camel-case naming style. const string& camelcase_name() const; - Type type() const; // Declared type of this field. - CppType cpp_type() const; // C++ type of this field. - Label label() const; // optional/required/repeated + Type type() const; // Declared type of this field. + const char* type_name() const; // Name of the declared type. + CppType cpp_type() const; // C++ type of this field. + const char* cpp_type_name() const; // Name of the C++ type. + Label label() const; // optional/required/repeated bool is_required() const; // shorthand for label() == LABEL_REQUIRED bool is_optional() const; // shorthand for label() == LABEL_OPTIONAL bool is_repeated() const; // shorthand for label() == LABEL_REPEATED bool is_packable() const; // shorthand for is_repeated() && // IsTypePackable(type()) + bool is_packed() const; // shorthand for is_packable() && + // options().packed() // Index of this field within the message's field array, or the file or // extension scope's extensions array. @@ -479,6 +509,13 @@ class LIBPROTOBUF_EXPORT FieldDescriptor { // Return true iff [packed = true] is valid for fields of this type. static inline bool IsTypePackable(Type field_type); + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this field declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef FieldOptions OptionsType; @@ -490,6 +527,10 @@ class LIBPROTOBUF_EXPORT FieldDescriptor { // types of CPPTYPE_STRING whill be surrounded by quotes and CEscaped. string DefaultValueAsString(bool quote_string_type) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; const string* lowercase_name_; @@ -527,6 +568,8 @@ class LIBPROTOBUF_EXPORT FieldDescriptor { static const char * const kTypeToName[MAX_TYPE + 1]; + static const char * const kCppTypeToName[MAX_CPPTYPE + 1]; + static const char * const kLabelToName[MAX_LABEL + 1]; // Must be constructed using DescriptorPool. @@ -583,12 +626,23 @@ class LIBPROTOBUF_EXPORT EnumDescriptor { // See Descriptor::DebugString(). string DebugString() const; + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this enum declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef EnumOptions OptionsType; // See Descriptor::DebugString(). void DebugString(int depth, string *contents) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; const FileDescriptor* file_; @@ -650,12 +704,23 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptor { // See Descriptor::DebugString(). string DebugString() const; + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this enum value declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef EnumValueOptions OptionsType; // See Descriptor::DebugString(). void DebugString(int depth, string *contents) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; int number_; @@ -703,19 +768,29 @@ class LIBPROTOBUF_EXPORT ServiceDescriptor { // Look up a MethodDescriptor by name. const MethodDescriptor* FindMethodByName(const string& name) const; - // See Descriptor::CopyTo(). void CopyTo(ServiceDescriptorProto* proto) const; // See Descriptor::DebugString(). string DebugString() const; + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this service declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef ServiceOptions OptionsType; // See Descriptor::DebugString(). void DebugString(string *contents) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; const FileDescriptor* file_; @@ -768,12 +843,23 @@ class LIBPROTOBUF_EXPORT MethodDescriptor { // See Descriptor::DebugString(). string DebugString() const; + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of this method declaration. Returns false and leaves + // |*out_location| unchanged iff location information was not available. + bool GetSourceLocation(SourceLocation* out_location) const; + private: typedef MethodOptions OptionsType; // See Descriptor::DebugString(). void DebugString(int depth, string *contents) const; + // Walks up the descriptor tree to generate the source location path + // to this descriptor from the file root. + void GetLocationPath(vector<int>* output) const; + const string* name_; const string* full_name_; const ServiceDescriptor* service_; @@ -791,6 +877,7 @@ class LIBPROTOBUF_EXPORT MethodDescriptor { GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(MethodDescriptor); }; + // Describes a whole .proto file. To get the FileDescriptor for a compiled-in // file, get the descriptor for something defined in that file and call // descriptor->file(). Use DescriptorPool to construct your own descriptors. @@ -813,6 +900,22 @@ class LIBPROTOBUF_EXPORT FileDescriptor { // These are returned in the order they were defined in the .proto file. const FileDescriptor* dependency(int index) const; + // The number of files public imported by this one. + // The public dependency list is a subset of the dependency list. + int public_dependency_count() const; + // Gets a public imported file by index, where 0 <= index < + // public_dependency_count(). + // These are returned in the order they were defined in the .proto file. + const FileDescriptor* public_dependency(int index) const; + + // The number of files that are imported for weak fields. + // The weak dependency list is a subset of the dependency list. + int weak_dependency_count() const; + // Gets a weak imported file by index, where 0 <= index < + // weak_dependency_count(). + // These are returned in the order they were defined in the .proto file. + const FileDescriptor* weak_dependency(int index) const; + // Number of top-level message types defined in this file. (This does not // include nested types.) int message_type_count() const; @@ -866,12 +969,28 @@ class LIBPROTOBUF_EXPORT FileDescriptor { const FieldDescriptor* FindExtensionByCamelcaseName(const string& name) const; // See Descriptor::CopyTo(). + // Notes: + // - This method does NOT copy source code information since it is relatively + // large and rarely needed. See CopySourceCodeInfoTo() below. void CopyTo(FileDescriptorProto* proto) const; + // Write the source code information of this FileDescriptor into the given + // FileDescriptorProto. See CopyTo() above. + void CopySourceCodeInfoTo(FileDescriptorProto* proto) const; // See Descriptor::DebugString(). string DebugString() const; private: + // Source Location --------------------------------------------------- + + // Updates |*out_location| to the source location of the complete + // extent of the declaration or declaration-part denoted by |path|. + // Returns false and leaves |*out_location| unchanged iff location + // information was not available. (See SourceCodeInfo for + // description of path encoding.) + bool GetSourceLocation(const vector<int>& path, + SourceLocation* out_location) const; + typedef FileOptions OptionsType; const string* name_; @@ -879,6 +998,10 @@ class LIBPROTOBUF_EXPORT FileDescriptor { const DescriptorPool* pool_; int dependency_count_; const FileDescriptor** dependencies_; + int public_dependency_count_; + int* public_dependencies_; + int weak_dependency_count_; + int* weak_dependencies_; int message_type_count_; Descriptor* message_types_; int enum_type_count_; @@ -890,6 +1013,7 @@ class LIBPROTOBUF_EXPORT FileDescriptor { const FileOptions* options_; const FileDescriptorTables* tables_; + const SourceCodeInfo* source_code_info_; // IMPORTANT: If you add a new field, make sure to search for all instances // of Allocate<FileDescriptor>() and AllocateArray<FileDescriptor>() in // descriptor.cc and update them to initialize the field. @@ -899,6 +1023,8 @@ class LIBPROTOBUF_EXPORT FileDescriptor { friend class Descriptor; friend class FieldDescriptor; friend class EnumDescriptor; + friend class EnumValueDescriptor; + friend class MethodDescriptor; friend class ServiceDescriptor; GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(FileDescriptor); }; @@ -1106,7 +1232,7 @@ class LIBPROTOBUF_EXPORT DescriptorPool { // For internal use only: Gets a non-const pointer to the generated pool. // This is called at static-initialization time only, so thread-safety is // not a concern. If both an underlay and a fallback database are present, - // the fallback database takes precedence. + // the underlay takes precedence. static DescriptorPool* internal_generated_pool(); // For internal use only: Changes the behavior of BuildFile() such that it @@ -1132,6 +1258,11 @@ class LIBPROTOBUF_EXPORT DescriptorPool { friend class FileDescriptor; friend class DescriptorBuilder; + // Return true if the given name is a sub-symbol of any non-package + // descriptor that already exists in the descriptor pool. (The full + // definition of such types is already known.) + bool IsSubSymbolOfBuiltType(const string& name) const; + // Tries to find something in the fallback database and link in the // corresponding proto file. Returns true if successful, in which case // the caller should search for the thing again. These are declared @@ -1262,11 +1393,12 @@ PROTOBUF_DEFINE_ACCESSOR(MethodDescriptor, service, const ServiceDescriptor*) PROTOBUF_DEFINE_ACCESSOR(MethodDescriptor, input_type, const Descriptor*) PROTOBUF_DEFINE_ACCESSOR(MethodDescriptor, output_type, const Descriptor*) PROTOBUF_DEFINE_OPTIONS_ACCESSOR(MethodDescriptor, MethodOptions); - PROTOBUF_DEFINE_STRING_ACCESSOR(FileDescriptor, name) PROTOBUF_DEFINE_STRING_ACCESSOR(FileDescriptor, package) PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, pool, const DescriptorPool*) PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, dependency_count, int) +PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, public_dependency_count, int) +PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, weak_dependency_count, int) PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, message_type_count, int) PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, enum_type_count, int) PROTOBUF_DEFINE_ACCESSOR(FileDescriptor, service_count, int) @@ -1342,10 +1474,18 @@ inline int MethodDescriptor::index() const { return this - service_->methods_; } +inline const char* FieldDescriptor::type_name() const { + return kTypeToName[type_]; +} + inline FieldDescriptor::CppType FieldDescriptor::cpp_type() const { return kTypeToCppTypeMap[type_]; } +inline const char* FieldDescriptor::cpp_type_name() const { + return kCppTypeToName[kTypeToCppTypeMap[type_]]; +} + inline FieldDescriptor::CppType FieldDescriptor::TypeToCppType(Type type) { return kTypeToCppTypeMap[type]; } @@ -1361,6 +1501,16 @@ inline const FileDescriptor* FileDescriptor::dependency(int index) const { return dependencies_[index]; } +inline const FileDescriptor* FileDescriptor::public_dependency( + int index) const { + return dependencies_[public_dependencies_[index]]; +} + +inline const FileDescriptor* FileDescriptor::weak_dependency( + int index) const { + return dependencies_[weak_dependencies_[index]]; +} + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/descriptor.pb.cc b/third_party/protobuf/src/google/protobuf/descriptor.pb.cc index 7d16080..a05414a 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor.pb.cc +++ b/third_party/protobuf/src/google/protobuf/descriptor.pb.cc @@ -1,14 +1,17 @@ // Generated by the protocol buffer compiler. DO NOT EDIT! +// source: google/protobuf/descriptor.proto #define INTERNAL_SUPPRESS_PROTOBUF_FIELD_DEPRECATION #include "google/protobuf/descriptor.pb.h" #include <algorithm> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/stubs/once.h> #include <google/protobuf/io/coded_stream.h> #include <google/protobuf/wire_format_lite_inl.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/generated_message_reflection.h> #include <google/protobuf/reflection_ops.h> #include <google/protobuf/wire_format.h> // @@protoc_insertion_point(includes) @@ -108,10 +111,12 @@ void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() { ::google::protobuf::MessageFactory::generated_factory(), sizeof(FileDescriptorSet)); FileDescriptorProto_descriptor_ = file->message_type(1); - static const int FileDescriptorProto_offsets_[9] = { + static const int FileDescriptorProto_offsets_[11] = { GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, name_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, package_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, dependency_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, public_dependency_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, weak_dependency_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, message_type_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, enum_type_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileDescriptorProto, service_), @@ -261,13 +266,14 @@ void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() { ::google::protobuf::MessageFactory::generated_factory(), sizeof(MethodDescriptorProto)); FileOptions_descriptor_ = file->message_type(8); - static const int FileOptions_offsets_[10] = { + static const int FileOptions_offsets_[11] = { GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, java_package_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, java_outer_classname_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, java_multiple_files_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, retain_unknown_fields_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, java_generate_equals_and_hash_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, optimize_for_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, go_package_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, cc_generic_services_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, java_generic_services_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FileOptions, py_generic_services_), @@ -303,11 +309,13 @@ void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() { ::google::protobuf::MessageFactory::generated_factory(), sizeof(MessageOptions)); FieldOptions_descriptor_ = file->message_type(10); - static const int FieldOptions_offsets_[5] = { + static const int FieldOptions_offsets_[7] = { GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, ctype_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, packed_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, lazy_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, deprecated_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, experimental_map_key_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, weak_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(FieldOptions, uninterpreted_option_), }; FieldOptions_reflection_ = @@ -323,7 +331,8 @@ void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() { sizeof(FieldOptions)); FieldOptions_CType_descriptor_ = FieldOptions_descriptor_->enum_type(0); EnumOptions_descriptor_ = file->message_type(11); - static const int EnumOptions_offsets_[1] = { + static const int EnumOptions_offsets_[2] = { + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(EnumOptions, allow_alias_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(EnumOptions, uninterpreted_option_), }; EnumOptions_reflection_ = @@ -435,9 +444,11 @@ void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() { ::google::protobuf::MessageFactory::generated_factory(), sizeof(SourceCodeInfo)); SourceCodeInfo_Location_descriptor_ = SourceCodeInfo_descriptor_->nested_type(0); - static const int SourceCodeInfo_Location_offsets_[2] = { + static const int SourceCodeInfo_Location_offsets_[4] = { GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(SourceCodeInfo_Location, path_), GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(SourceCodeInfo_Location, span_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(SourceCodeInfo_Location, leading_comments_), + GOOGLE_PROTOBUF_GENERATED_MESSAGE_FIELD_OFFSET(SourceCodeInfo_Location, trailing_comments_), }; SourceCodeInfo_Location_reflection_ = new ::google::protobuf::internal::GeneratedMessageReflection( @@ -554,107 +565,113 @@ void protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto() { if (already_here) return; already_here = true; GOOGLE_PROTOBUF_VERIFY_VERSION; + ::google::protobuf::DescriptorPool::InternalAddGeneratedFile( "\n google/protobuf/descriptor.proto\022\017goog" "le.protobuf\"G\n\021FileDescriptorSet\0222\n\004file" "\030\001 \003(\0132$.google.protobuf.FileDescriptorP" - "roto\"\227\003\n\023FileDescriptorProto\022\014\n\004name\030\001 \001" + "roto\"\313\003\n\023FileDescriptorProto\022\014\n\004name\030\001 \001" "(\t\022\017\n\007package\030\002 \001(\t\022\022\n\ndependency\030\003 \003(\t\022" - "6\n\014message_type\030\004 \003(\0132 .google.protobuf." - "DescriptorProto\0227\n\tenum_type\030\005 \003(\0132$.goo" - "gle.protobuf.EnumDescriptorProto\0228\n\007serv" - "ice\030\006 \003(\0132\'.google.protobuf.ServiceDescr" - "iptorProto\0228\n\textension\030\007 \003(\0132%.google.p" - "rotobuf.FieldDescriptorProto\022-\n\007options\030" - "\010 \001(\0132\034.google.protobuf.FileOptions\0229\n\020s" - "ource_code_info\030\t \001(\0132\037.google.protobuf." - "SourceCodeInfo\"\251\003\n\017DescriptorProto\022\014\n\004na" - "me\030\001 \001(\t\0224\n\005field\030\002 \003(\0132%.google.protobu" - "f.FieldDescriptorProto\0228\n\textension\030\006 \003(" + "\031\n\021public_dependency\030\n \003(\005\022\027\n\017weak_depen" + "dency\030\013 \003(\005\0226\n\014message_type\030\004 \003(\0132 .goog" + "le.protobuf.DescriptorProto\0227\n\tenum_type" + "\030\005 \003(\0132$.google.protobuf.EnumDescriptorP" + "roto\0228\n\007service\030\006 \003(\0132\'.google.protobuf." + "ServiceDescriptorProto\0228\n\textension\030\007 \003(" "\0132%.google.protobuf.FieldDescriptorProto" - "\0225\n\013nested_type\030\003 \003(\0132 .google.protobuf." - "DescriptorProto\0227\n\tenum_type\030\004 \003(\0132$.goo" - "gle.protobuf.EnumDescriptorProto\022H\n\017exte" - "nsion_range\030\005 \003(\0132/.google.protobuf.Desc" - "riptorProto.ExtensionRange\0220\n\007options\030\007 " - "\001(\0132\037.google.protobuf.MessageOptions\032,\n\016" - "ExtensionRange\022\r\n\005start\030\001 \001(\005\022\013\n\003end\030\002 \001" - "(\005\"\224\005\n\024FieldDescriptorProto\022\014\n\004name\030\001 \001(" - "\t\022\016\n\006number\030\003 \001(\005\022:\n\005label\030\004 \001(\0162+.googl" - "e.protobuf.FieldDescriptorProto.Label\0228\n" - "\004type\030\005 \001(\0162*.google.protobuf.FieldDescr" - "iptorProto.Type\022\021\n\ttype_name\030\006 \001(\t\022\020\n\010ex" - "tendee\030\002 \001(\t\022\025\n\rdefault_value\030\007 \001(\t\022.\n\007o" - "ptions\030\010 \001(\0132\035.google.protobuf.FieldOpti" - "ons\"\266\002\n\004Type\022\017\n\013TYPE_DOUBLE\020\001\022\016\n\nTYPE_FL" - "OAT\020\002\022\016\n\nTYPE_INT64\020\003\022\017\n\013TYPE_UINT64\020\004\022\016" - "\n\nTYPE_INT32\020\005\022\020\n\014TYPE_FIXED64\020\006\022\020\n\014TYPE" - "_FIXED32\020\007\022\r\n\tTYPE_BOOL\020\010\022\017\n\013TYPE_STRING" - "\020\t\022\016\n\nTYPE_GROUP\020\n\022\020\n\014TYPE_MESSAGE\020\013\022\016\n\n" - "TYPE_BYTES\020\014\022\017\n\013TYPE_UINT32\020\r\022\r\n\tTYPE_EN" - "UM\020\016\022\021\n\rTYPE_SFIXED32\020\017\022\021\n\rTYPE_SFIXED64" - "\020\020\022\017\n\013TYPE_SINT32\020\021\022\017\n\013TYPE_SINT64\020\022\"C\n\005" - "Label\022\022\n\016LABEL_OPTIONAL\020\001\022\022\n\016LABEL_REQUI" - "RED\020\002\022\022\n\016LABEL_REPEATED\020\003\"\214\001\n\023EnumDescri" - "ptorProto\022\014\n\004name\030\001 \001(\t\0228\n\005value\030\002 \003(\0132)" - ".google.protobuf.EnumValueDescriptorProt" - "o\022-\n\007options\030\003 \001(\0132\034.google.protobuf.Enu" - "mOptions\"l\n\030EnumValueDescriptorProto\022\014\n\004" - "name\030\001 \001(\t\022\016\n\006number\030\002 \001(\005\0222\n\007options\030\003 " - "\001(\0132!.google.protobuf.EnumValueOptions\"\220" - "\001\n\026ServiceDescriptorProto\022\014\n\004name\030\001 \001(\t\022" - "6\n\006method\030\002 \003(\0132&.google.protobuf.Method" - "DescriptorProto\0220\n\007options\030\003 \001(\0132\037.googl" - "e.protobuf.ServiceOptions\"\177\n\025MethodDescr" - "iptorProto\022\014\n\004name\030\001 \001(\t\022\022\n\ninput_type\030\002" - " \001(\t\022\023\n\013output_type\030\003 \001(\t\022/\n\007options\030\004 \001" - "(\0132\036.google.protobuf.MethodOptions\"\373\003\n\013F" - "ileOptions\022\024\n\014java_package\030\001 \001(\t\022\034\n\024java" - "_outer_classname\030\010 \001(\t\022\"\n\023java_multiple_" - "files\030\n \001(\010:\005false\022$\n\025retain_unknown_fie" - "lds\030\013 \001(\010:\005false\022,\n\035java_generate_equals" - "_and_hash\030\024 \001(\010:\005false\022F\n\014optimize_for\030\t" - " \001(\0162).google.protobuf.FileOptions.Optim" - "izeMode:\005SPEED\022\"\n\023cc_generic_services\030\020 " - "\001(\010:\005false\022$\n\025java_generic_services\030\021 \001(" - "\010:\005false\022\"\n\023py_generic_services\030\022 \001(\010:\005f" - "alse\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.go" - "ogle.protobuf.UninterpretedOption\":\n\014Opt" - "imizeMode\022\t\n\005SPEED\020\001\022\r\n\tCODE_SIZE\020\002\022\020\n\014L" - "ITE_RUNTIME\020\003*\t\010\350\007\020\200\200\200\200\002\"\270\001\n\016MessageOpti" - "ons\022&\n\027message_set_wire_format\030\001 \001(\010:\005fa" - "lse\022.\n\037no_standard_descriptor_accessor\030\002" - " \001(\010:\005false\022C\n\024uninterpreted_option\030\347\007 \003" - "(\0132$.google.protobuf.UninterpretedOption" - "*\t\010\350\007\020\200\200\200\200\002\"\224\002\n\014FieldOptions\022:\n\005ctype\030\001 " - "\001(\0162#.google.protobuf.FieldOptions.CType" - ":\006STRING\022\016\n\006packed\030\002 \001(\010\022\031\n\ndeprecated\030\003" - " \001(\010:\005false\022\034\n\024experimental_map_key\030\t \001(" - "\t\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.googl" - "e.protobuf.UninterpretedOption\"/\n\005CType\022" - "\n\n\006STRING\020\000\022\010\n\004CORD\020\001\022\020\n\014STRING_PIECE\020\002*" - "\t\010\350\007\020\200\200\200\200\002\"]\n\013EnumOptions\022C\n\024uninterpret" - "ed_option\030\347\007 \003(\0132$.google.protobuf.Unint" - "erpretedOption*\t\010\350\007\020\200\200\200\200\002\"b\n\020EnumValueOp" - "tions\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.g" - "oogle.protobuf.UninterpretedOption*\t\010\350\007\020" - "\200\200\200\200\002\"`\n\016ServiceOptions\022C\n\024uninterpreted" - "_option\030\347\007 \003(\0132$.google.protobuf.Uninter" - "pretedOption*\t\010\350\007\020\200\200\200\200\002\"_\n\rMethodOptions" - "\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.google" - ".protobuf.UninterpretedOption*\t\010\350\007\020\200\200\200\200\002" - "\"\236\002\n\023UninterpretedOption\022;\n\004name\030\002 \003(\0132-" - ".google.protobuf.UninterpretedOption.Nam" - "ePart\022\030\n\020identifier_value\030\003 \001(\t\022\032\n\022posit" - "ive_int_value\030\004 \001(\004\022\032\n\022negative_int_valu" - "e\030\005 \001(\003\022\024\n\014double_value\030\006 \001(\001\022\024\n\014string_" - "value\030\007 \001(\014\022\027\n\017aggregate_value\030\010 \001(\t\0323\n\010" - "NamePart\022\021\n\tname_part\030\001 \002(\t\022\024\n\014is_extens" - "ion\030\002 \002(\010\"|\n\016SourceCodeInfo\022:\n\010location\030" - "\001 \003(\0132(.google.protobuf.SourceCodeInfo.L" - "ocation\032.\n\010Location\022\020\n\004path\030\001 \003(\005B\002\020\001\022\020\n" - "\004span\030\002 \003(\005B\002\020\001B)\n\023com.google.protobufB\020" - "DescriptorProtosH\001", 3978); + "\022-\n\007options\030\010 \001(\0132\034.google.protobuf.File" + "Options\0229\n\020source_code_info\030\t \001(\0132\037.goog" + "le.protobuf.SourceCodeInfo\"\251\003\n\017Descripto" + "rProto\022\014\n\004name\030\001 \001(\t\0224\n\005field\030\002 \003(\0132%.go" + "ogle.protobuf.FieldDescriptorProto\0228\n\tex" + "tension\030\006 \003(\0132%.google.protobuf.FieldDes" + "criptorProto\0225\n\013nested_type\030\003 \003(\0132 .goog" + "le.protobuf.DescriptorProto\0227\n\tenum_type" + "\030\004 \003(\0132$.google.protobuf.EnumDescriptorP" + "roto\022H\n\017extension_range\030\005 \003(\0132/.google.p" + "rotobuf.DescriptorProto.ExtensionRange\0220" + "\n\007options\030\007 \001(\0132\037.google.protobuf.Messag" + "eOptions\032,\n\016ExtensionRange\022\r\n\005start\030\001 \001(" + "\005\022\013\n\003end\030\002 \001(\005\"\224\005\n\024FieldDescriptorProto\022" + "\014\n\004name\030\001 \001(\t\022\016\n\006number\030\003 \001(\005\022:\n\005label\030\004" + " \001(\0162+.google.protobuf.FieldDescriptorPr" + "oto.Label\0228\n\004type\030\005 \001(\0162*.google.protobu" + "f.FieldDescriptorProto.Type\022\021\n\ttype_name" + "\030\006 \001(\t\022\020\n\010extendee\030\002 \001(\t\022\025\n\rdefault_valu" + "e\030\007 \001(\t\022.\n\007options\030\010 \001(\0132\035.google.protob" + "uf.FieldOptions\"\266\002\n\004Type\022\017\n\013TYPE_DOUBLE\020" + "\001\022\016\n\nTYPE_FLOAT\020\002\022\016\n\nTYPE_INT64\020\003\022\017\n\013TYP" + "E_UINT64\020\004\022\016\n\nTYPE_INT32\020\005\022\020\n\014TYPE_FIXED" + "64\020\006\022\020\n\014TYPE_FIXED32\020\007\022\r\n\tTYPE_BOOL\020\010\022\017\n" + "\013TYPE_STRING\020\t\022\016\n\nTYPE_GROUP\020\n\022\020\n\014TYPE_M" + "ESSAGE\020\013\022\016\n\nTYPE_BYTES\020\014\022\017\n\013TYPE_UINT32\020" + "\r\022\r\n\tTYPE_ENUM\020\016\022\021\n\rTYPE_SFIXED32\020\017\022\021\n\rT" + "YPE_SFIXED64\020\020\022\017\n\013TYPE_SINT32\020\021\022\017\n\013TYPE_" + "SINT64\020\022\"C\n\005Label\022\022\n\016LABEL_OPTIONAL\020\001\022\022\n" + "\016LABEL_REQUIRED\020\002\022\022\n\016LABEL_REPEATED\020\003\"\214\001" + "\n\023EnumDescriptorProto\022\014\n\004name\030\001 \001(\t\0228\n\005v" + "alue\030\002 \003(\0132).google.protobuf.EnumValueDe" + "scriptorProto\022-\n\007options\030\003 \001(\0132\034.google." + "protobuf.EnumOptions\"l\n\030EnumValueDescrip" + "torProto\022\014\n\004name\030\001 \001(\t\022\016\n\006number\030\002 \001(\005\0222" + "\n\007options\030\003 \001(\0132!.google.protobuf.EnumVa" + "lueOptions\"\220\001\n\026ServiceDescriptorProto\022\014\n" + "\004name\030\001 \001(\t\0226\n\006method\030\002 \003(\0132&.google.pro" + "tobuf.MethodDescriptorProto\0220\n\007options\030\003" + " \001(\0132\037.google.protobuf.ServiceOptions\"\177\n" + "\025MethodDescriptorProto\022\014\n\004name\030\001 \001(\t\022\022\n\n" + "input_type\030\002 \001(\t\022\023\n\013output_type\030\003 \001(\t\022/\n" + "\007options\030\004 \001(\0132\036.google.protobuf.MethodO" + "ptions\"\217\004\n\013FileOptions\022\024\n\014java_package\030\001" + " \001(\t\022\034\n\024java_outer_classname\030\010 \001(\t\022\"\n\023ja" + "va_multiple_files\030\n \001(\010:\005false\022$\n\025retain" + "_unknown_fields\030\014 \001(\010:\005false\022,\n\035java_gen" + "erate_equals_and_hash\030\024 \001(\010:\005false\022F\n\014op" + "timize_for\030\t \001(\0162).google.protobuf.FileO" + "ptions.OptimizeMode:\005SPEED\022\022\n\ngo_package" + "\030\013 \001(\t\022\"\n\023cc_generic_services\030\020 \001(\010:\005fal" + "se\022$\n\025java_generic_services\030\021 \001(\010:\005false" + "\022\"\n\023py_generic_services\030\022 \001(\010:\005false\022C\n\024" + "uninterpreted_option\030\347\007 \003(\0132$.google.pro" + "tobuf.UninterpretedOption\":\n\014OptimizeMod" + "e\022\t\n\005SPEED\020\001\022\r\n\tCODE_SIZE\020\002\022\020\n\014LITE_RUNT" + "IME\020\003*\t\010\350\007\020\200\200\200\200\002\"\270\001\n\016MessageOptions\022&\n\027m" + "essage_set_wire_format\030\001 \001(\010:\005false\022.\n\037n" + "o_standard_descriptor_accessor\030\002 \001(\010:\005fa" + "lse\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.goo" + "gle.protobuf.UninterpretedOption*\t\010\350\007\020\200\200" + "\200\200\002\"\276\002\n\014FieldOptions\022:\n\005ctype\030\001 \001(\0162#.go" + "ogle.protobuf.FieldOptions.CType:\006STRING" + "\022\016\n\006packed\030\002 \001(\010\022\023\n\004lazy\030\005 \001(\010:\005false\022\031\n" + "\ndeprecated\030\003 \001(\010:\005false\022\034\n\024experimental" + "_map_key\030\t \001(\t\022\023\n\004weak\030\n \001(\010:\005false\022C\n\024u" + "ninterpreted_option\030\347\007 \003(\0132$.google.prot" + "obuf.UninterpretedOption\"/\n\005CType\022\n\n\006STR" + "ING\020\000\022\010\n\004CORD\020\001\022\020\n\014STRING_PIECE\020\002*\t\010\350\007\020\200" + "\200\200\200\002\"x\n\013EnumOptions\022\031\n\013allow_alias\030\002 \001(\010" + ":\004true\022C\n\024uninterpreted_option\030\347\007 \003(\0132$." + "google.protobuf.UninterpretedOption*\t\010\350\007" + "\020\200\200\200\200\002\"b\n\020EnumValueOptions\022C\n\024uninterpre" + "ted_option\030\347\007 \003(\0132$.google.protobuf.Unin" + "terpretedOption*\t\010\350\007\020\200\200\200\200\002\"`\n\016ServiceOpt" + "ions\022C\n\024uninterpreted_option\030\347\007 \003(\0132$.go" + "ogle.protobuf.UninterpretedOption*\t\010\350\007\020\200" + "\200\200\200\002\"_\n\rMethodOptions\022C\n\024uninterpreted_o" + "ption\030\347\007 \003(\0132$.google.protobuf.Uninterpr" + "etedOption*\t\010\350\007\020\200\200\200\200\002\"\236\002\n\023UninterpretedO" + "ption\022;\n\004name\030\002 \003(\0132-.google.protobuf.Un" + "interpretedOption.NamePart\022\030\n\020identifier" + "_value\030\003 \001(\t\022\032\n\022positive_int_value\030\004 \001(\004" + "\022\032\n\022negative_int_value\030\005 \001(\003\022\024\n\014double_v" + "alue\030\006 \001(\001\022\024\n\014string_value\030\007 \001(\014\022\027\n\017aggr" + "egate_value\030\010 \001(\t\0323\n\010NamePart\022\021\n\tname_pa" + "rt\030\001 \002(\t\022\024\n\014is_extension\030\002 \002(\010\"\261\001\n\016Sourc" + "eCodeInfo\022:\n\010location\030\001 \003(\0132(.google.pro" + "tobuf.SourceCodeInfo.Location\032c\n\010Locatio" + "n\022\020\n\004path\030\001 \003(\005B\002\020\001\022\020\n\004span\030\002 \003(\005B\002\020\001\022\030\n" + "\020leading_comments\030\003 \001(\t\022\031\n\021trailing_comm" + "ents\030\004 \001(\tB)\n\023com.google.protobufB\020Descr" + "iptorProtosH\001", 4173); ::google::protobuf::MessageFactory::InternalRegisterGeneratedFile( "google/protobuf/descriptor.proto", &protobuf_RegisterTypes); FileDescriptorSet::default_instance_ = new FileDescriptorSet(); @@ -788,14 +805,15 @@ bool FileDescriptorSet::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -811,7 +829,7 @@ void FileDescriptorSet::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 1, this->file(i), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -826,7 +844,7 @@ void FileDescriptorSet::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 1, this->file(i), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -836,7 +854,7 @@ void FileDescriptorSet::SerializeWithCachedSizes( int FileDescriptorSet::ByteSize() const { int total_size = 0; - + // repeated .google.protobuf.FileDescriptorProto file = 1; total_size += 1 * this->file_size(); for (int i = 0; i < this->file_size(); i++) { @@ -844,7 +862,7 @@ int FileDescriptorSet::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->file(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -887,7 +905,7 @@ void FileDescriptorSet::CopyFrom(const FileDescriptorSet& from) { } bool FileDescriptorSet::IsInitialized() const { - + for (int i = 0; i < file_size(); i++) { if (!this->file(i).IsInitialized()) return false; } @@ -918,6 +936,8 @@ void FileDescriptorSet::Swap(FileDescriptorSet* other) { const int FileDescriptorProto::kNameFieldNumber; const int FileDescriptorProto::kPackageFieldNumber; const int FileDescriptorProto::kDependencyFieldNumber; +const int FileDescriptorProto::kPublicDependencyFieldNumber; +const int FileDescriptorProto::kWeakDependencyFieldNumber; const int FileDescriptorProto::kMessageTypeFieldNumber; const int FileDescriptorProto::kEnumTypeFieldNumber; const int FileDescriptorProto::kServiceFieldNumber; @@ -1001,16 +1021,18 @@ void FileDescriptorProto::Clear() { package_->clear(); } } + } + if (_has_bits_[9 / 32] & (0xffu << (9 % 32))) { if (has_options()) { if (options_ != NULL) options_->::google::protobuf::FileOptions::Clear(); } - } - if (_has_bits_[8 / 32] & (0xffu << (8 % 32))) { if (has_source_code_info()) { if (source_code_info_ != NULL) source_code_info_->::google::protobuf::SourceCodeInfo::Clear(); } } dependency_.Clear(); + public_dependency_.Clear(); + weak_dependency_.Clear(); message_type_.Clear(); enum_type_.Clear(); service_.Clear(); @@ -1040,7 +1062,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_package; break; } - + // optional string package = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1057,7 +1079,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_dependency; break; } - + // repeated string dependency = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1076,7 +1098,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(34)) goto parse_message_type; break; } - + // repeated .google.protobuf.DescriptorProto message_type = 4; case 4: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1091,7 +1113,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(42)) goto parse_enum_type; break; } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 5; case 5: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1106,7 +1128,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(50)) goto parse_service; break; } - + // repeated .google.protobuf.ServiceDescriptorProto service = 6; case 6: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1121,7 +1143,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(58)) goto parse_extension; break; } - + // repeated .google.protobuf.FieldDescriptorProto extension = 7; case 7: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1136,7 +1158,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(66)) goto parse_options; break; } - + // optional .google.protobuf.FileOptions options = 8; case 8: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1150,7 +1172,7 @@ bool FileDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(74)) goto parse_source_code_info; break; } - + // optional .google.protobuf.SourceCodeInfo source_code_info = 9; case 9: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1161,17 +1183,62 @@ bool FileDescriptorProto::MergePartialFromCodedStream( } else { goto handle_uninterpreted; } + if (input->ExpectTag(80)) goto parse_public_dependency; + break; + } + + // repeated int32 public_dependency = 10; + case 10: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { + parse_public_dependency: + DO_((::google::protobuf::internal::WireFormatLite::ReadRepeatedPrimitive< + ::google::protobuf::int32, ::google::protobuf::internal::WireFormatLite::TYPE_INT32>( + 1, 80, input, this->mutable_public_dependency()))); + } else if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) + == ::google::protobuf::internal::WireFormatLite:: + WIRETYPE_LENGTH_DELIMITED) { + DO_((::google::protobuf::internal::WireFormatLite::ReadPackedPrimitiveNoInline< + ::google::protobuf::int32, ::google::protobuf::internal::WireFormatLite::TYPE_INT32>( + input, this->mutable_public_dependency()))); + } else { + goto handle_uninterpreted; + } + if (input->ExpectTag(80)) goto parse_public_dependency; + if (input->ExpectTag(88)) goto parse_weak_dependency; + break; + } + + // repeated int32 weak_dependency = 11; + case 11: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { + parse_weak_dependency: + DO_((::google::protobuf::internal::WireFormatLite::ReadRepeatedPrimitive< + ::google::protobuf::int32, ::google::protobuf::internal::WireFormatLite::TYPE_INT32>( + 1, 88, input, this->mutable_weak_dependency()))); + } else if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) + == ::google::protobuf::internal::WireFormatLite:: + WIRETYPE_LENGTH_DELIMITED) { + DO_((::google::protobuf::internal::WireFormatLite::ReadPackedPrimitiveNoInline< + ::google::protobuf::int32, ::google::protobuf::internal::WireFormatLite::TYPE_INT32>( + input, this->mutable_weak_dependency()))); + } else { + goto handle_uninterpreted; + } + if (input->ExpectTag(88)) goto parse_weak_dependency; if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -1190,7 +1257,7 @@ void FileDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // optional string package = 2; if (has_package()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -1199,7 +1266,7 @@ void FileDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 2, this->package(), output); } - + // repeated string dependency = 3; for (int i = 0; i < this->dependency_size(); i++) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -1208,43 +1275,55 @@ void FileDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 3, this->dependency(i), output); } - + // repeated .google.protobuf.DescriptorProto message_type = 4; for (int i = 0; i < this->message_type_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 4, this->message_type(i), output); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 5; for (int i = 0; i < this->enum_type_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 5, this->enum_type(i), output); } - + // repeated .google.protobuf.ServiceDescriptorProto service = 6; for (int i = 0; i < this->service_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 6, this->service(i), output); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 7; for (int i = 0; i < this->extension_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 7, this->extension(i), output); } - + // optional .google.protobuf.FileOptions options = 8; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 8, this->options(), output); } - + // optional .google.protobuf.SourceCodeInfo source_code_info = 9; if (has_source_code_info()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 9, this->source_code_info(), output); } - + + // repeated int32 public_dependency = 10; + for (int i = 0; i < this->public_dependency_size(); i++) { + ::google::protobuf::internal::WireFormatLite::WriteInt32( + 10, this->public_dependency(i), output); + } + + // repeated int32 weak_dependency = 11; + for (int i = 0; i < this->weak_dependency_size(); i++) { + ::google::protobuf::internal::WireFormatLite::WriteInt32( + 11, this->weak_dependency(i), output); + } + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -1262,7 +1341,7 @@ void FileDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // optional string package = 2; if (has_package()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -1272,7 +1351,7 @@ void FileDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 2, this->package(), target); } - + // repeated string dependency = 3; for (int i = 0; i < this->dependency_size(); i++) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -1281,49 +1360,61 @@ void FileDescriptorProto::SerializeWithCachedSizes( target = ::google::protobuf::internal::WireFormatLite:: WriteStringToArray(3, this->dependency(i), target); } - + // repeated .google.protobuf.DescriptorProto message_type = 4; for (int i = 0; i < this->message_type_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 4, this->message_type(i), target); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 5; for (int i = 0; i < this->enum_type_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 5, this->enum_type(i), target); } - + // repeated .google.protobuf.ServiceDescriptorProto service = 6; for (int i = 0; i < this->service_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 6, this->service(i), target); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 7; for (int i = 0; i < this->extension_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 7, this->extension(i), target); } - + // optional .google.protobuf.FileOptions options = 8; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 8, this->options(), target); } - + // optional .google.protobuf.SourceCodeInfo source_code_info = 9; if (has_source_code_info()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 9, this->source_code_info(), target); } - + + // repeated int32 public_dependency = 10; + for (int i = 0; i < this->public_dependency_size(); i++) { + target = ::google::protobuf::internal::WireFormatLite:: + WriteInt32ToArray(10, this->public_dependency(i), target); + } + + // repeated int32 weak_dependency = 11; + for (int i = 0; i < this->weak_dependency_size(); i++) { + target = ::google::protobuf::internal::WireFormatLite:: + WriteInt32ToArray(11, this->weak_dependency(i), target); + } + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -1333,7 +1424,7 @@ void FileDescriptorProto::SerializeWithCachedSizes( int FileDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -1341,30 +1432,30 @@ int FileDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional string package = 2; if (has_package()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->package()); } - + + } + if (_has_bits_[9 / 32] & (0xffu << (9 % 32))) { // optional .google.protobuf.FileOptions options = 8; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - - } - if (_has_bits_[8 / 32] & (0xffu << (8 % 32))) { + // optional .google.protobuf.SourceCodeInfo source_code_info = 9; if (has_source_code_info()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->source_code_info()); } - + } // repeated string dependency = 3; total_size += 1 * this->dependency_size(); @@ -1372,7 +1463,27 @@ int FileDescriptorProto::ByteSize() const { total_size += ::google::protobuf::internal::WireFormatLite::StringSize( this->dependency(i)); } - + + // repeated int32 public_dependency = 10; + { + int data_size = 0; + for (int i = 0; i < this->public_dependency_size(); i++) { + data_size += ::google::protobuf::internal::WireFormatLite:: + Int32Size(this->public_dependency(i)); + } + total_size += 1 * this->public_dependency_size() + data_size; + } + + // repeated int32 weak_dependency = 11; + { + int data_size = 0; + for (int i = 0; i < this->weak_dependency_size(); i++) { + data_size += ::google::protobuf::internal::WireFormatLite:: + Int32Size(this->weak_dependency(i)); + } + total_size += 1 * this->weak_dependency_size() + data_size; + } + // repeated .google.protobuf.DescriptorProto message_type = 4; total_size += 1 * this->message_type_size(); for (int i = 0; i < this->message_type_size(); i++) { @@ -1380,7 +1491,7 @@ int FileDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->message_type(i)); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 5; total_size += 1 * this->enum_type_size(); for (int i = 0; i < this->enum_type_size(); i++) { @@ -1388,7 +1499,7 @@ int FileDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->enum_type(i)); } - + // repeated .google.protobuf.ServiceDescriptorProto service = 6; total_size += 1 * this->service_size(); for (int i = 0; i < this->service_size(); i++) { @@ -1396,7 +1507,7 @@ int FileDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->service(i)); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 7; total_size += 1 * this->extension_size(); for (int i = 0; i < this->extension_size(); i++) { @@ -1404,7 +1515,7 @@ int FileDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->extension(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -1431,6 +1542,8 @@ void FileDescriptorProto::MergeFrom(const ::google::protobuf::Message& from) { void FileDescriptorProto::MergeFrom(const FileDescriptorProto& from) { GOOGLE_CHECK_NE(&from, this); dependency_.MergeFrom(from.dependency_); + public_dependency_.MergeFrom(from.public_dependency_); + weak_dependency_.MergeFrom(from.weak_dependency_); message_type_.MergeFrom(from.message_type_); enum_type_.MergeFrom(from.enum_type_); service_.MergeFrom(from.service_); @@ -1442,11 +1555,11 @@ void FileDescriptorProto::MergeFrom(const FileDescriptorProto& from) { if (from.has_package()) { set_package(from.package()); } + } + if (from._has_bits_[9 / 32] & (0xffu << (9 % 32))) { if (from.has_options()) { mutable_options()->::google::protobuf::FileOptions::MergeFrom(from.options()); } - } - if (from._has_bits_[8 / 32] & (0xffu << (8 % 32))) { if (from.has_source_code_info()) { mutable_source_code_info()->::google::protobuf::SourceCodeInfo::MergeFrom(from.source_code_info()); } @@ -1467,7 +1580,7 @@ void FileDescriptorProto::CopyFrom(const FileDescriptorProto& from) { } bool FileDescriptorProto::IsInitialized() const { - + for (int i = 0; i < message_type_size(); i++) { if (!this->message_type(i).IsInitialized()) return false; } @@ -1491,6 +1604,8 @@ void FileDescriptorProto::Swap(FileDescriptorProto* other) { std::swap(name_, other->name_); std::swap(package_, other->package_); dependency_.Swap(&other->dependency_); + public_dependency_.Swap(&other->public_dependency_); + weak_dependency_.Swap(&other->weak_dependency_); message_type_.Swap(&other->message_type_); enum_type_.Swap(&other->enum_type_); service_.Swap(&other->service_); @@ -1599,7 +1714,7 @@ bool DescriptorProto_ExtensionRange::MergePartialFromCodedStream( if (input->ExpectTag(16)) goto parse_end; break; } - + // optional int32 end = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1615,14 +1730,15 @@ bool DescriptorProto_ExtensionRange::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -1637,12 +1753,12 @@ void DescriptorProto_ExtensionRange::SerializeWithCachedSizes( if (has_start()) { ::google::protobuf::internal::WireFormatLite::WriteInt32(1, this->start(), output); } - + // optional int32 end = 2; if (has_end()) { ::google::protobuf::internal::WireFormatLite::WriteInt32(2, this->end(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -1655,12 +1771,12 @@ void DescriptorProto_ExtensionRange::SerializeWithCachedSizes( if (has_start()) { target = ::google::protobuf::internal::WireFormatLite::WriteInt32ToArray(1, this->start(), target); } - + // optional int32 end = 2; if (has_end()) { target = ::google::protobuf::internal::WireFormatLite::WriteInt32ToArray(2, this->end(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -1670,7 +1786,7 @@ void DescriptorProto_ExtensionRange::SerializeWithCachedSizes( int DescriptorProto_ExtensionRange::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional int32 start = 1; if (has_start()) { @@ -1678,14 +1794,14 @@ int DescriptorProto_ExtensionRange::ByteSize() const { ::google::protobuf::internal::WireFormatLite::Int32Size( this->start()); } - + // optional int32 end = 2; if (has_end()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int32Size( this->end()); } - + } if (!unknown_fields().empty()) { total_size += @@ -1736,7 +1852,7 @@ void DescriptorProto_ExtensionRange::CopyFrom(const DescriptorProto_ExtensionRan } bool DescriptorProto_ExtensionRange::IsInitialized() const { - + return true; } @@ -1868,7 +1984,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_field; break; } - + // repeated .google.protobuf.FieldDescriptorProto field = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1883,7 +1999,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_nested_type; break; } - + // repeated .google.protobuf.DescriptorProto nested_type = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1898,7 +2014,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(34)) goto parse_enum_type; break; } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 4; case 4: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1913,7 +2029,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(42)) goto parse_extension_range; break; } - + // repeated .google.protobuf.DescriptorProto.ExtensionRange extension_range = 5; case 5: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1928,7 +2044,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(50)) goto parse_extension; break; } - + // repeated .google.protobuf.FieldDescriptorProto extension = 6; case 6: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1943,7 +2059,7 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(58)) goto parse_options; break; } - + // optional .google.protobuf.MessageOptions options = 7; case 7: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -1957,14 +2073,15 @@ bool DescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -1983,43 +2100,43 @@ void DescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // repeated .google.protobuf.FieldDescriptorProto field = 2; for (int i = 0; i < this->field_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 2, this->field(i), output); } - + // repeated .google.protobuf.DescriptorProto nested_type = 3; for (int i = 0; i < this->nested_type_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 3, this->nested_type(i), output); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 4; for (int i = 0; i < this->enum_type_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 4, this->enum_type(i), output); } - + // repeated .google.protobuf.DescriptorProto.ExtensionRange extension_range = 5; for (int i = 0; i < this->extension_range_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 5, this->extension_range(i), output); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 6; for (int i = 0; i < this->extension_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 6, this->extension(i), output); } - + // optional .google.protobuf.MessageOptions options = 7; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 7, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -2037,49 +2154,49 @@ void DescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // repeated .google.protobuf.FieldDescriptorProto field = 2; for (int i = 0; i < this->field_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 2, this->field(i), target); } - + // repeated .google.protobuf.DescriptorProto nested_type = 3; for (int i = 0; i < this->nested_type_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 3, this->nested_type(i), target); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 4; for (int i = 0; i < this->enum_type_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 4, this->enum_type(i), target); } - + // repeated .google.protobuf.DescriptorProto.ExtensionRange extension_range = 5; for (int i = 0; i < this->extension_range_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 5, this->extension_range(i), target); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 6; for (int i = 0; i < this->extension_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 6, this->extension(i), target); } - + // optional .google.protobuf.MessageOptions options = 7; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 7, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -2089,7 +2206,7 @@ void DescriptorProto::SerializeWithCachedSizes( int DescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -2097,14 +2214,14 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional .google.protobuf.MessageOptions options = 7; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } // repeated .google.protobuf.FieldDescriptorProto field = 2; total_size += 1 * this->field_size(); @@ -2113,7 +2230,7 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->field(i)); } - + // repeated .google.protobuf.FieldDescriptorProto extension = 6; total_size += 1 * this->extension_size(); for (int i = 0; i < this->extension_size(); i++) { @@ -2121,7 +2238,7 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->extension(i)); } - + // repeated .google.protobuf.DescriptorProto nested_type = 3; total_size += 1 * this->nested_type_size(); for (int i = 0; i < this->nested_type_size(); i++) { @@ -2129,7 +2246,7 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->nested_type(i)); } - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 4; total_size += 1 * this->enum_type_size(); for (int i = 0; i < this->enum_type_size(); i++) { @@ -2137,7 +2254,7 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->enum_type(i)); } - + // repeated .google.protobuf.DescriptorProto.ExtensionRange extension_range = 5; total_size += 1 * this->extension_range_size(); for (int i = 0; i < this->extension_range_size(); i++) { @@ -2145,7 +2262,7 @@ int DescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->extension_range(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -2200,7 +2317,7 @@ void DescriptorProto::CopyFrom(const DescriptorProto& from) { } bool DescriptorProto::IsInitialized() const { - + for (int i = 0; i < field_size(); i++) { if (!this->field(i).IsInitialized()) return false; } @@ -2457,7 +2574,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_extendee; break; } - + // optional string extendee = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2474,7 +2591,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(24)) goto parse_number; break; } - + // optional int32 number = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2490,7 +2607,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(32)) goto parse_label; break; } - + // optional .google.protobuf.FieldDescriptorProto.Label label = 4; case 4: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2511,7 +2628,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(40)) goto parse_type; break; } - + // optional .google.protobuf.FieldDescriptorProto.Type type = 5; case 5: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2532,7 +2649,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(50)) goto parse_type_name; break; } - + // optional string type_name = 6; case 6: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2549,7 +2666,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(58)) goto parse_default_value; break; } - + // optional string default_value = 7; case 7: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2566,7 +2683,7 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(66)) goto parse_options; break; } - + // optional .google.protobuf.FieldOptions options = 8; case 8: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -2580,14 +2697,15 @@ bool FieldDescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -2606,7 +2724,7 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // optional string extendee = 2; if (has_extendee()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2615,24 +2733,24 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 2, this->extendee(), output); } - + // optional int32 number = 3; if (has_number()) { ::google::protobuf::internal::WireFormatLite::WriteInt32(3, this->number(), output); } - + // optional .google.protobuf.FieldDescriptorProto.Label label = 4; if (has_label()) { ::google::protobuf::internal::WireFormatLite::WriteEnum( 4, this->label(), output); } - + // optional .google.protobuf.FieldDescriptorProto.Type type = 5; if (has_type()) { ::google::protobuf::internal::WireFormatLite::WriteEnum( 5, this->type(), output); } - + // optional string type_name = 6; if (has_type_name()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2641,7 +2759,7 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 6, this->type_name(), output); } - + // optional string default_value = 7; if (has_default_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2650,13 +2768,13 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 7, this->default_value(), output); } - + // optional .google.protobuf.FieldOptions options = 8; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 8, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -2674,7 +2792,7 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // optional string extendee = 2; if (has_extendee()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2684,24 +2802,24 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 2, this->extendee(), target); } - + // optional int32 number = 3; if (has_number()) { target = ::google::protobuf::internal::WireFormatLite::WriteInt32ToArray(3, this->number(), target); } - + // optional .google.protobuf.FieldDescriptorProto.Label label = 4; if (has_label()) { target = ::google::protobuf::internal::WireFormatLite::WriteEnumToArray( 4, this->label(), target); } - + // optional .google.protobuf.FieldDescriptorProto.Type type = 5; if (has_type()) { target = ::google::protobuf::internal::WireFormatLite::WriteEnumToArray( 5, this->type(), target); } - + // optional string type_name = 6; if (has_type_name()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2711,7 +2829,7 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 6, this->type_name(), target); } - + // optional string default_value = 7; if (has_default_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -2721,14 +2839,14 @@ void FieldDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 7, this->default_value(), target); } - + // optional .google.protobuf.FieldOptions options = 8; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 8, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -2738,7 +2856,7 @@ void FieldDescriptorProto::SerializeWithCachedSizes( int FieldDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -2746,54 +2864,54 @@ int FieldDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional int32 number = 3; if (has_number()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int32Size( this->number()); } - + // optional .google.protobuf.FieldDescriptorProto.Label label = 4; if (has_label()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::EnumSize(this->label()); } - + // optional .google.protobuf.FieldDescriptorProto.Type type = 5; if (has_type()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::EnumSize(this->type()); } - + // optional string type_name = 6; if (has_type_name()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->type_name()); } - + // optional string extendee = 2; if (has_extendee()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->extendee()); } - + // optional string default_value = 7; if (has_default_value()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->default_value()); } - + // optional .google.protobuf.FieldOptions options = 8; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } if (!unknown_fields().empty()) { total_size += @@ -2862,7 +2980,7 @@ void FieldDescriptorProto::CopyFrom(const FieldDescriptorProto& from) { } bool FieldDescriptorProto::IsInitialized() const { - + if (has_options()) { if (!this->options().IsInitialized()) return false; } @@ -2995,7 +3113,7 @@ bool EnumDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_value; break; } - + // repeated .google.protobuf.EnumValueDescriptorProto value = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3010,7 +3128,7 @@ bool EnumDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_options; break; } - + // optional .google.protobuf.EnumOptions options = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3024,14 +3142,15 @@ bool EnumDescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -3050,19 +3169,19 @@ void EnumDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // repeated .google.protobuf.EnumValueDescriptorProto value = 2; for (int i = 0; i < this->value_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 2, this->value(i), output); } - + // optional .google.protobuf.EnumOptions options = 3; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 3, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -3080,21 +3199,21 @@ void EnumDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // repeated .google.protobuf.EnumValueDescriptorProto value = 2; for (int i = 0; i < this->value_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 2, this->value(i), target); } - + // optional .google.protobuf.EnumOptions options = 3; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 3, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -3104,7 +3223,7 @@ void EnumDescriptorProto::SerializeWithCachedSizes( int EnumDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -3112,14 +3231,14 @@ int EnumDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional .google.protobuf.EnumOptions options = 3; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } // repeated .google.protobuf.EnumValueDescriptorProto value = 2; total_size += 1 * this->value_size(); @@ -3128,7 +3247,7 @@ int EnumDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->value(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -3179,7 +3298,7 @@ void EnumDescriptorProto::CopyFrom(const EnumDescriptorProto& from) { } bool EnumDescriptorProto::IsInitialized() const { - + for (int i = 0; i < value_size(); i++) { if (!this->value(i).IsInitialized()) return false; } @@ -3311,7 +3430,7 @@ bool EnumValueDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(16)) goto parse_number; break; } - + // optional int32 number = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3327,7 +3446,7 @@ bool EnumValueDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_options; break; } - + // optional .google.protobuf.EnumValueOptions options = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3341,14 +3460,15 @@ bool EnumValueDescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -3367,18 +3487,18 @@ void EnumValueDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // optional int32 number = 2; if (has_number()) { ::google::protobuf::internal::WireFormatLite::WriteInt32(2, this->number(), output); } - + // optional .google.protobuf.EnumValueOptions options = 3; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 3, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -3396,19 +3516,19 @@ void EnumValueDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // optional int32 number = 2; if (has_number()) { target = ::google::protobuf::internal::WireFormatLite::WriteInt32ToArray(2, this->number(), target); } - + // optional .google.protobuf.EnumValueOptions options = 3; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 3, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -3418,7 +3538,7 @@ void EnumValueDescriptorProto::SerializeWithCachedSizes( int EnumValueDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -3426,21 +3546,21 @@ int EnumValueDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional int32 number = 2; if (has_number()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int32Size( this->number()); } - + // optional .google.protobuf.EnumValueOptions options = 3; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } if (!unknown_fields().empty()) { total_size += @@ -3494,7 +3614,7 @@ void EnumValueDescriptorProto::CopyFrom(const EnumValueDescriptorProto& from) { } bool EnumValueDescriptorProto::IsInitialized() const { - + if (has_options()) { if (!this->options().IsInitialized()) return false; } @@ -3622,7 +3742,7 @@ bool ServiceDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_method; break; } - + // repeated .google.protobuf.MethodDescriptorProto method = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3637,7 +3757,7 @@ bool ServiceDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_options; break; } - + // optional .google.protobuf.ServiceOptions options = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3651,14 +3771,15 @@ bool ServiceDescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -3677,19 +3798,19 @@ void ServiceDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // repeated .google.protobuf.MethodDescriptorProto method = 2; for (int i = 0; i < this->method_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 2, this->method(i), output); } - + // optional .google.protobuf.ServiceOptions options = 3; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 3, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -3707,21 +3828,21 @@ void ServiceDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // repeated .google.protobuf.MethodDescriptorProto method = 2; for (int i = 0; i < this->method_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 2, this->method(i), target); } - + // optional .google.protobuf.ServiceOptions options = 3; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 3, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -3731,7 +3852,7 @@ void ServiceDescriptorProto::SerializeWithCachedSizes( int ServiceDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -3739,14 +3860,14 @@ int ServiceDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional .google.protobuf.ServiceOptions options = 3; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } // repeated .google.protobuf.MethodDescriptorProto method = 2; total_size += 1 * this->method_size(); @@ -3755,7 +3876,7 @@ int ServiceDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->method(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -3806,7 +3927,7 @@ void ServiceDescriptorProto::CopyFrom(const ServiceDescriptorProto& from) { } bool ServiceDescriptorProto::IsInitialized() const { - + for (int i = 0; i < method_size(); i++) { if (!this->method(i).IsInitialized()) return false; } @@ -3955,7 +4076,7 @@ bool MethodDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_input_type; break; } - + // optional string input_type = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3972,7 +4093,7 @@ bool MethodDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_output_type; break; } - + // optional string output_type = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -3989,7 +4110,7 @@ bool MethodDescriptorProto::MergePartialFromCodedStream( if (input->ExpectTag(34)) goto parse_options; break; } - + // optional .google.protobuf.MethodOptions options = 4; case 4: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4003,14 +4124,15 @@ bool MethodDescriptorProto::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -4029,7 +4151,7 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name(), output); } - + // optional string input_type = 2; if (has_input_type()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4038,7 +4160,7 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 2, this->input_type(), output); } - + // optional string output_type = 3; if (has_output_type()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4047,13 +4169,13 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 3, this->output_type(), output); } - + // optional .google.protobuf.MethodOptions options = 4; if (has_options()) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 4, this->options(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -4071,7 +4193,7 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name(), target); } - + // optional string input_type = 2; if (has_input_type()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4081,7 +4203,7 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 2, this->input_type(), target); } - + // optional string output_type = 3; if (has_output_type()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4091,14 +4213,14 @@ void MethodDescriptorProto::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 3, this->output_type(), target); } - + // optional .google.protobuf.MethodOptions options = 4; if (has_options()) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 4, this->options(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -4108,7 +4230,7 @@ void MethodDescriptorProto::SerializeWithCachedSizes( int MethodDescriptorProto::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string name = 1; if (has_name()) { @@ -4116,28 +4238,28 @@ int MethodDescriptorProto::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name()); } - + // optional string input_type = 2; if (has_input_type()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->input_type()); } - + // optional string output_type = 3; if (has_output_type()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->output_type()); } - + // optional .google.protobuf.MethodOptions options = 4; if (has_options()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->options()); } - + } if (!unknown_fields().empty()) { total_size += @@ -4194,7 +4316,7 @@ void MethodDescriptorProto::CopyFrom(const MethodDescriptorProto& from) { } bool MethodDescriptorProto::IsInitialized() const { - + if (has_options()) { if (!this->options().IsInitialized()) return false; } @@ -4254,6 +4376,7 @@ const int FileOptions::kJavaMultipleFilesFieldNumber; const int FileOptions::kRetainUnknownFieldsFieldNumber; const int FileOptions::kJavaGenerateEqualsAndHashFieldNumber; const int FileOptions::kOptimizeForFieldNumber; +const int FileOptions::kGoPackageFieldNumber; const int FileOptions::kCcGenericServicesFieldNumber; const int FileOptions::kJavaGenericServicesFieldNumber; const int FileOptions::kPyGenericServicesFieldNumber; @@ -4282,6 +4405,7 @@ void FileOptions::SharedCtor() { retain_unknown_fields_ = false; java_generate_equals_and_hash_ = false; optimize_for_ = 1; + go_package_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); cc_generic_services_ = false; java_generic_services_ = false; py_generic_services_ = false; @@ -4299,6 +4423,9 @@ void FileOptions::SharedDtor() { if (java_outer_classname_ != &::google::protobuf::internal::kEmptyString) { delete java_outer_classname_; } + if (go_package_ != &::google::protobuf::internal::kEmptyString) { + delete go_package_; + } if (this != default_instance_) { } } @@ -4341,10 +4468,15 @@ void FileOptions::Clear() { retain_unknown_fields_ = false; java_generate_equals_and_hash_ = false; optimize_for_ = 1; + if (has_go_package()) { + if (go_package_ != &::google::protobuf::internal::kEmptyString) { + go_package_->clear(); + } + } cc_generic_services_ = false; - java_generic_services_ = false; } if (_has_bits_[8 / 32] & (0xffu << (8 % 32))) { + java_generic_services_ = false; py_generic_services_ = false; } uninterpreted_option_.Clear(); @@ -4373,7 +4505,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(66)) goto parse_java_outer_classname; break; } - + // optional string java_outer_classname = 8; case 8: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4390,7 +4522,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(72)) goto parse_optimize_for; break; } - + // optional .google.protobuf.FileOptions.OptimizeMode optimize_for = 9 [default = SPEED]; case 9: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4411,7 +4543,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(80)) goto parse_java_multiple_files; break; } - + // optional bool java_multiple_files = 10 [default = false]; case 10: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4424,13 +4556,30 @@ bool FileOptions::MergePartialFromCodedStream( } else { goto handle_uninterpreted; } - if (input->ExpectTag(88)) goto parse_retain_unknown_fields; + if (input->ExpectTag(90)) goto parse_go_package; break; } - - // optional bool retain_unknown_fields = 11 [default = false]; + + // optional string go_package = 11; case 11: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED) { + parse_go_package: + DO_(::google::protobuf::internal::WireFormatLite::ReadString( + input, this->mutable_go_package())); + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->go_package().data(), this->go_package().length(), + ::google::protobuf::internal::WireFormat::PARSE); + } else { + goto handle_uninterpreted; + } + if (input->ExpectTag(96)) goto parse_retain_unknown_fields; + break; + } + + // optional bool retain_unknown_fields = 12 [default = false]; + case 12: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { parse_retain_unknown_fields: DO_((::google::protobuf::internal::WireFormatLite::ReadPrimitive< @@ -4443,7 +4592,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(128)) goto parse_cc_generic_services; break; } - + // optional bool cc_generic_services = 16 [default = false]; case 16: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4459,7 +4608,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(136)) goto parse_java_generic_services; break; } - + // optional bool java_generic_services = 17 [default = false]; case 17: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4475,7 +4624,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(144)) goto parse_py_generic_services; break; } - + // optional bool py_generic_services = 18 [default = false]; case 18: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4491,7 +4640,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(160)) goto parse_java_generate_equals_and_hash; break; } - + // optional bool java_generate_equals_and_hash = 20 [default = false]; case 20: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4507,7 +4656,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectTag(7994)) goto parse_uninterpreted_option; break; } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; case 999: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4522,7 +4671,7 @@ bool FileOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4534,7 +4683,8 @@ bool FileOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -4553,7 +4703,7 @@ void FileOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->java_package(), output); } - + // optional string java_outer_classname = 8; if (has_java_outer_classname()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4562,53 +4712,62 @@ void FileOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 8, this->java_outer_classname(), output); } - + // optional .google.protobuf.FileOptions.OptimizeMode optimize_for = 9 [default = SPEED]; if (has_optimize_for()) { ::google::protobuf::internal::WireFormatLite::WriteEnum( 9, this->optimize_for(), output); } - + // optional bool java_multiple_files = 10 [default = false]; if (has_java_multiple_files()) { ::google::protobuf::internal::WireFormatLite::WriteBool(10, this->java_multiple_files(), output); } - - // optional bool retain_unknown_fields = 11 [default = false]; + + // optional string go_package = 11; + if (has_go_package()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->go_package().data(), this->go_package().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + ::google::protobuf::internal::WireFormatLite::WriteString( + 11, this->go_package(), output); + } + + // optional bool retain_unknown_fields = 12 [default = false]; if (has_retain_unknown_fields()) { - ::google::protobuf::internal::WireFormatLite::WriteBool(11, this->retain_unknown_fields(), output); + ::google::protobuf::internal::WireFormatLite::WriteBool(12, this->retain_unknown_fields(), output); } - + // optional bool cc_generic_services = 16 [default = false]; if (has_cc_generic_services()) { ::google::protobuf::internal::WireFormatLite::WriteBool(16, this->cc_generic_services(), output); } - + // optional bool java_generic_services = 17 [default = false]; if (has_java_generic_services()) { ::google::protobuf::internal::WireFormatLite::WriteBool(17, this->java_generic_services(), output); } - + // optional bool py_generic_services = 18 [default = false]; if (has_py_generic_services()) { ::google::protobuf::internal::WireFormatLite::WriteBool(18, this->py_generic_services(), output); } - + // optional bool java_generate_equals_and_hash = 20 [default = false]; if (has_java_generate_equals_and_hash()) { ::google::protobuf::internal::WireFormatLite::WriteBool(20, this->java_generate_equals_and_hash(), output); } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -4626,7 +4785,7 @@ void FileOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->java_package(), target); } - + // optional string java_outer_classname = 8; if (has_java_outer_classname()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -4636,54 +4795,64 @@ void FileOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 8, this->java_outer_classname(), target); } - + // optional .google.protobuf.FileOptions.OptimizeMode optimize_for = 9 [default = SPEED]; if (has_optimize_for()) { target = ::google::protobuf::internal::WireFormatLite::WriteEnumToArray( 9, this->optimize_for(), target); } - + // optional bool java_multiple_files = 10 [default = false]; if (has_java_multiple_files()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(10, this->java_multiple_files(), target); } - - // optional bool retain_unknown_fields = 11 [default = false]; + + // optional string go_package = 11; + if (has_go_package()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->go_package().data(), this->go_package().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + target = + ::google::protobuf::internal::WireFormatLite::WriteStringToArray( + 11, this->go_package(), target); + } + + // optional bool retain_unknown_fields = 12 [default = false]; if (has_retain_unknown_fields()) { - target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(11, this->retain_unknown_fields(), target); + target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(12, this->retain_unknown_fields(), target); } - + // optional bool cc_generic_services = 16 [default = false]; if (has_cc_generic_services()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(16, this->cc_generic_services(), target); } - + // optional bool java_generic_services = 17 [default = false]; if (has_java_generic_services()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(17, this->java_generic_services(), target); } - + // optional bool py_generic_services = 18 [default = false]; if (has_py_generic_services()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(18, this->py_generic_services(), target); } - + // optional bool java_generate_equals_and_hash = 20 [default = false]; if (has_java_generate_equals_and_hash()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(20, this->java_generate_equals_and_hash(), target); } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -4693,7 +4862,7 @@ void FileOptions::SerializeWithCachedSizes( int FileOptions::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional string java_package = 1; if (has_java_package()) { @@ -4701,52 +4870,59 @@ int FileOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->java_package()); } - + // optional string java_outer_classname = 8; if (has_java_outer_classname()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->java_outer_classname()); } - + // optional bool java_multiple_files = 10 [default = false]; if (has_java_multiple_files()) { total_size += 1 + 1; } - - // optional bool retain_unknown_fields = 11 [default = false]; + + // optional bool retain_unknown_fields = 12 [default = false]; if (has_retain_unknown_fields()) { total_size += 1 + 1; } - + // optional bool java_generate_equals_and_hash = 20 [default = false]; if (has_java_generate_equals_and_hash()) { total_size += 2 + 1; } - + // optional .google.protobuf.FileOptions.OptimizeMode optimize_for = 9 [default = SPEED]; if (has_optimize_for()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::EnumSize(this->optimize_for()); } - + + // optional string go_package = 11; + if (has_go_package()) { + total_size += 1 + + ::google::protobuf::internal::WireFormatLite::StringSize( + this->go_package()); + } + // optional bool cc_generic_services = 16 [default = false]; if (has_cc_generic_services()) { total_size += 2 + 1; } - + + } + if (_has_bits_[8 / 32] & (0xffu << (8 % 32))) { // optional bool java_generic_services = 17 [default = false]; if (has_java_generic_services()) { total_size += 2 + 1; } - - } - if (_has_bits_[8 / 32] & (0xffu << (8 % 32))) { + // optional bool py_generic_services = 18 [default = false]; if (has_py_generic_services()) { total_size += 2 + 1; } - + } // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); @@ -4755,9 +4931,9 @@ int FileOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -4803,14 +4979,17 @@ void FileOptions::MergeFrom(const FileOptions& from) { if (from.has_optimize_for()) { set_optimize_for(from.optimize_for()); } + if (from.has_go_package()) { + set_go_package(from.go_package()); + } if (from.has_cc_generic_services()) { set_cc_generic_services(from.cc_generic_services()); } + } + if (from._has_bits_[8 / 32] & (0xffu << (8 % 32))) { if (from.has_java_generic_services()) { set_java_generic_services(from.java_generic_services()); } - } - if (from._has_bits_[8 / 32] & (0xffu << (8 % 32))) { if (from.has_py_generic_services()) { set_py_generic_services(from.py_generic_services()); } @@ -4832,11 +5011,11 @@ void FileOptions::CopyFrom(const FileOptions& from) { } bool FileOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -4848,6 +5027,7 @@ void FileOptions::Swap(FileOptions* other) { std::swap(retain_unknown_fields_, other->retain_unknown_fields_); std::swap(java_generate_equals_and_hash_, other->java_generate_equals_and_hash_); std::swap(optimize_for_, other->optimize_for_); + std::swap(go_package_, other->go_package_); std::swap(cc_generic_services_, other->cc_generic_services_); std::swap(java_generic_services_, other->java_generic_services_); std::swap(py_generic_services_, other->py_generic_services_); @@ -4958,7 +5138,7 @@ bool MessageOptions::MergePartialFromCodedStream( if (input->ExpectTag(16)) goto parse_no_standard_descriptor_accessor; break; } - + // optional bool no_standard_descriptor_accessor = 2 [default = false]; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4974,7 +5154,7 @@ bool MessageOptions::MergePartialFromCodedStream( if (input->ExpectTag(7994)) goto parse_uninterpreted_option; break; } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; case 999: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -4989,7 +5169,7 @@ bool MessageOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5001,7 +5181,8 @@ bool MessageOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -5016,22 +5197,22 @@ void MessageOptions::SerializeWithCachedSizes( if (has_message_set_wire_format()) { ::google::protobuf::internal::WireFormatLite::WriteBool(1, this->message_set_wire_format(), output); } - + // optional bool no_standard_descriptor_accessor = 2 [default = false]; if (has_no_standard_descriptor_accessor()) { ::google::protobuf::internal::WireFormatLite::WriteBool(2, this->no_standard_descriptor_accessor(), output); } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -5044,23 +5225,23 @@ void MessageOptions::SerializeWithCachedSizes( if (has_message_set_wire_format()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(1, this->message_set_wire_format(), target); } - + // optional bool no_standard_descriptor_accessor = 2 [default = false]; if (has_no_standard_descriptor_accessor()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(2, this->no_standard_descriptor_accessor(), target); } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -5070,18 +5251,18 @@ void MessageOptions::SerializeWithCachedSizes( int MessageOptions::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional bool message_set_wire_format = 1 [default = false]; if (has_message_set_wire_format()) { total_size += 1 + 1; } - + // optional bool no_standard_descriptor_accessor = 2 [default = false]; if (has_no_standard_descriptor_accessor()) { total_size += 1 + 1; } - + } // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); @@ -5090,9 +5271,9 @@ int MessageOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -5144,11 +5325,11 @@ void MessageOptions::CopyFrom(const MessageOptions& from) { } bool MessageOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -5201,8 +5382,10 @@ const int FieldOptions::CType_ARRAYSIZE; #ifndef _MSC_VER const int FieldOptions::kCtypeFieldNumber; const int FieldOptions::kPackedFieldNumber; +const int FieldOptions::kLazyFieldNumber; const int FieldOptions::kDeprecatedFieldNumber; const int FieldOptions::kExperimentalMapKeyFieldNumber; +const int FieldOptions::kWeakFieldNumber; const int FieldOptions::kUninterpretedOptionFieldNumber; #endif // !_MSC_VER @@ -5224,8 +5407,10 @@ void FieldOptions::SharedCtor() { _cached_size_ = 0; ctype_ = 0; packed_ = false; + lazy_ = false; deprecated_ = false; experimental_map_key_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + weak_ = false; ::memset(_has_bits_, 0, sizeof(_has_bits_)); } @@ -5267,12 +5452,14 @@ void FieldOptions::Clear() { if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { ctype_ = 0; packed_ = false; + lazy_ = false; deprecated_ = false; if (has_experimental_map_key()) { if (experimental_map_key_ != &::google::protobuf::internal::kEmptyString) { experimental_map_key_->clear(); } } + weak_ = false; } uninterpreted_option_.Clear(); ::memset(_has_bits_, 0, sizeof(_has_bits_)); @@ -5304,7 +5491,7 @@ bool FieldOptions::MergePartialFromCodedStream( if (input->ExpectTag(16)) goto parse_packed; break; } - + // optional bool packed = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5320,7 +5507,7 @@ bool FieldOptions::MergePartialFromCodedStream( if (input->ExpectTag(24)) goto parse_deprecated; break; } - + // optional bool deprecated = 3 [default = false]; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5333,10 +5520,26 @@ bool FieldOptions::MergePartialFromCodedStream( } else { goto handle_uninterpreted; } + if (input->ExpectTag(40)) goto parse_lazy; + break; + } + + // optional bool lazy = 5 [default = false]; + case 5: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { + parse_lazy: + DO_((::google::protobuf::internal::WireFormatLite::ReadPrimitive< + bool, ::google::protobuf::internal::WireFormatLite::TYPE_BOOL>( + input, &lazy_))); + set_has_lazy(); + } else { + goto handle_uninterpreted; + } if (input->ExpectTag(74)) goto parse_experimental_map_key; break; } - + // optional string experimental_map_key = 9; case 9: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5350,10 +5553,26 @@ bool FieldOptions::MergePartialFromCodedStream( } else { goto handle_uninterpreted; } + if (input->ExpectTag(80)) goto parse_weak; + break; + } + + // optional bool weak = 10 [default = false]; + case 10: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { + parse_weak: + DO_((::google::protobuf::internal::WireFormatLite::ReadPrimitive< + bool, ::google::protobuf::internal::WireFormatLite::TYPE_BOOL>( + input, &weak_))); + set_has_weak(); + } else { + goto handle_uninterpreted; + } if (input->ExpectTag(7994)) goto parse_uninterpreted_option; break; } - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; case 999: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5368,7 +5587,7 @@ bool FieldOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5380,7 +5599,8 @@ bool FieldOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -5396,17 +5616,22 @@ void FieldOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteEnum( 1, this->ctype(), output); } - + // optional bool packed = 2; if (has_packed()) { ::google::protobuf::internal::WireFormatLite::WriteBool(2, this->packed(), output); } - + // optional bool deprecated = 3 [default = false]; if (has_deprecated()) { ::google::protobuf::internal::WireFormatLite::WriteBool(3, this->deprecated(), output); } - + + // optional bool lazy = 5 [default = false]; + if (has_lazy()) { + ::google::protobuf::internal::WireFormatLite::WriteBool(5, this->lazy(), output); + } + // optional string experimental_map_key = 9; if (has_experimental_map_key()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -5415,17 +5640,22 @@ void FieldOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 9, this->experimental_map_key(), output); } - + + // optional bool weak = 10 [default = false]; + if (has_weak()) { + ::google::protobuf::internal::WireFormatLite::WriteBool(10, this->weak(), output); + } + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -5439,17 +5669,22 @@ void FieldOptions::SerializeWithCachedSizes( target = ::google::protobuf::internal::WireFormatLite::WriteEnumToArray( 1, this->ctype(), target); } - + // optional bool packed = 2; if (has_packed()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(2, this->packed(), target); } - + // optional bool deprecated = 3 [default = false]; if (has_deprecated()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(3, this->deprecated(), target); } - + + // optional bool lazy = 5 [default = false]; + if (has_lazy()) { + target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(5, this->lazy(), target); + } + // optional string experimental_map_key = 9; if (has_experimental_map_key()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -5459,18 +5694,23 @@ void FieldOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 9, this->experimental_map_key(), target); } - + + // optional bool weak = 10 [default = false]; + if (has_weak()) { + target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(10, this->weak(), target); + } + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -5480,31 +5720,41 @@ void FieldOptions::SerializeWithCachedSizes( int FieldOptions::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // optional .google.protobuf.FieldOptions.CType ctype = 1 [default = STRING]; if (has_ctype()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::EnumSize(this->ctype()); } - + // optional bool packed = 2; if (has_packed()) { total_size += 1 + 1; } - + + // optional bool lazy = 5 [default = false]; + if (has_lazy()) { + total_size += 1 + 1; + } + // optional bool deprecated = 3 [default = false]; if (has_deprecated()) { total_size += 1 + 1; } - + // optional string experimental_map_key = 9; if (has_experimental_map_key()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->experimental_map_key()); } - + + // optional bool weak = 10 [default = false]; + if (has_weak()) { + total_size += 1 + 1; + } + } // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); @@ -5513,9 +5763,9 @@ int FieldOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -5549,12 +5799,18 @@ void FieldOptions::MergeFrom(const FieldOptions& from) { if (from.has_packed()) { set_packed(from.packed()); } + if (from.has_lazy()) { + set_lazy(from.lazy()); + } if (from.has_deprecated()) { set_deprecated(from.deprecated()); } if (from.has_experimental_map_key()) { set_experimental_map_key(from.experimental_map_key()); } + if (from.has_weak()) { + set_weak(from.weak()); + } } _extensions_.MergeFrom(from._extensions_); mutable_unknown_fields()->MergeFrom(from.unknown_fields()); @@ -5573,11 +5829,11 @@ void FieldOptions::CopyFrom(const FieldOptions& from) { } bool FieldOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -5585,8 +5841,10 @@ void FieldOptions::Swap(FieldOptions* other) { if (other != this) { std::swap(ctype_, other->ctype_); std::swap(packed_, other->packed_); + std::swap(lazy_, other->lazy_); std::swap(deprecated_, other->deprecated_); std::swap(experimental_map_key_, other->experimental_map_key_); + std::swap(weak_, other->weak_); uninterpreted_option_.Swap(&other->uninterpreted_option_); std::swap(_has_bits_[0], other->_has_bits_[0]); _unknown_fields_.Swap(&other->_unknown_fields_); @@ -5607,6 +5865,7 @@ void FieldOptions::Swap(FieldOptions* other) { // =================================================================== #ifndef _MSC_VER +const int EnumOptions::kAllowAliasFieldNumber; const int EnumOptions::kUninterpretedOptionFieldNumber; #endif // !_MSC_VER @@ -5626,6 +5885,7 @@ EnumOptions::EnumOptions(const EnumOptions& from) void EnumOptions::SharedCtor() { _cached_size_ = 0; + allow_alias_ = true; ::memset(_has_bits_, 0, sizeof(_has_bits_)); } @@ -5661,6 +5921,9 @@ EnumOptions* EnumOptions::New() const { void EnumOptions::Clear() { _extensions_.Clear(); + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { + allow_alias_ = true; + } uninterpreted_option_.Clear(); ::memset(_has_bits_, 0, sizeof(_has_bits_)); mutable_unknown_fields()->Clear(); @@ -5672,6 +5935,21 @@ bool EnumOptions::MergePartialFromCodedStream( ::google::protobuf::uint32 tag; while ((tag = input->ReadTag()) != 0) { switch (::google::protobuf::internal::WireFormatLite::GetTagFieldNumber(tag)) { + // optional bool allow_alias = 2 [default = true]; + case 2: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_VARINT) { + DO_((::google::protobuf::internal::WireFormatLite::ReadPrimitive< + bool, ::google::protobuf::internal::WireFormatLite::TYPE_BOOL>( + input, &allow_alias_))); + set_has_allow_alias(); + } else { + goto handle_uninterpreted; + } + if (input->ExpectTag(7994)) goto parse_uninterpreted_option; + break; + } + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; case 999: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5686,7 +5964,7 @@ bool EnumOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5698,7 +5976,8 @@ bool EnumOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -5709,16 +5988,21 @@ bool EnumOptions::MergePartialFromCodedStream( void EnumOptions::SerializeWithCachedSizes( ::google::protobuf::io::CodedOutputStream* output) const { + // optional bool allow_alias = 2 [default = true]; + if (has_allow_alias()) { + ::google::protobuf::internal::WireFormatLite::WriteBool(2, this->allow_alias(), output); + } + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -5727,17 +6011,22 @@ void EnumOptions::SerializeWithCachedSizes( ::google::protobuf::uint8* EnumOptions::SerializeWithCachedSizesToArray( ::google::protobuf::uint8* target) const { + // optional bool allow_alias = 2 [default = true]; + if (has_allow_alias()) { + target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(2, this->allow_alias(), target); + } + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; for (int i = 0; i < this->uninterpreted_option_size(); i++) { target = ::google::protobuf::internal::WireFormatLite:: WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -5747,7 +6036,14 @@ void EnumOptions::SerializeWithCachedSizes( int EnumOptions::ByteSize() const { int total_size = 0; - + + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { + // optional bool allow_alias = 2 [default = true]; + if (has_allow_alias()) { + total_size += 1 + 1; + } + + } // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); for (int i = 0; i < this->uninterpreted_option_size(); i++) { @@ -5755,9 +6051,9 @@ int EnumOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -5784,6 +6080,11 @@ void EnumOptions::MergeFrom(const ::google::protobuf::Message& from) { void EnumOptions::MergeFrom(const EnumOptions& from) { GOOGLE_CHECK_NE(&from, this); uninterpreted_option_.MergeFrom(from.uninterpreted_option_); + if (from._has_bits_[0 / 32] & (0xffu << (0 % 32))) { + if (from.has_allow_alias()) { + set_allow_alias(from.allow_alias()); + } + } _extensions_.MergeFrom(from._extensions_); mutable_unknown_fields()->MergeFrom(from.unknown_fields()); } @@ -5801,16 +6102,17 @@ void EnumOptions::CopyFrom(const EnumOptions& from) { } bool EnumOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } void EnumOptions::Swap(EnumOptions* other) { if (other != this) { + std::swap(allow_alias_, other->allow_alias_); uninterpreted_option_.Swap(&other->uninterpreted_option_); std::swap(_has_bits_[0], other->_has_bits_[0]); _unknown_fields_.Swap(&other->_unknown_fields_); @@ -5910,7 +6212,7 @@ bool EnumValueOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -5922,7 +6224,8 @@ bool EnumValueOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -5938,11 +6241,11 @@ void EnumValueOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -5957,11 +6260,11 @@ void EnumValueOptions::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -5971,7 +6274,7 @@ void EnumValueOptions::SerializeWithCachedSizes( int EnumValueOptions::ByteSize() const { int total_size = 0; - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); for (int i = 0; i < this->uninterpreted_option_size(); i++) { @@ -5979,9 +6282,9 @@ int EnumValueOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -6025,11 +6328,11 @@ void EnumValueOptions::CopyFrom(const EnumValueOptions& from) { } bool EnumValueOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -6134,7 +6437,7 @@ bool ServiceOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6146,7 +6449,8 @@ bool ServiceOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -6162,11 +6466,11 @@ void ServiceOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -6181,11 +6485,11 @@ void ServiceOptions::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -6195,7 +6499,7 @@ void ServiceOptions::SerializeWithCachedSizes( int ServiceOptions::ByteSize() const { int total_size = 0; - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); for (int i = 0; i < this->uninterpreted_option_size(); i++) { @@ -6203,9 +6507,9 @@ int ServiceOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -6249,11 +6553,11 @@ void ServiceOptions::CopyFrom(const ServiceOptions& from) { } bool ServiceOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -6358,7 +6662,7 @@ bool MethodOptions::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6370,7 +6674,8 @@ bool MethodOptions::MergePartialFromCodedStream( mutable_unknown_fields())); continue; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -6386,11 +6691,11 @@ void MethodOptions::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 999, this->uninterpreted_option(i), output); } - + // Extension range [1000, 536870912) _extensions_.SerializeWithCachedSizes( 1000, 536870912, output); - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -6405,11 +6710,11 @@ void MethodOptions::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 999, this->uninterpreted_option(i), target); } - + // Extension range [1000, 536870912) target = _extensions_.SerializeWithCachedSizesToArray( 1000, 536870912, target); - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -6419,7 +6724,7 @@ void MethodOptions::SerializeWithCachedSizes( int MethodOptions::ByteSize() const { int total_size = 0; - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; total_size += 2 * this->uninterpreted_option_size(); for (int i = 0; i < this->uninterpreted_option_size(); i++) { @@ -6427,9 +6732,9 @@ int MethodOptions::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->uninterpreted_option(i)); } - + total_size += _extensions_.ByteSize(); - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -6473,11 +6778,11 @@ void MethodOptions::CopyFrom(const MethodOptions& from) { } bool MethodOptions::IsInitialized() const { - + for (int i = 0; i < uninterpreted_option_size(); i++) { if (!this->uninterpreted_option(i).IsInitialized()) return false; } - + if (!_extensions_.IsInitialized()) return false; return true; } @@ -6595,7 +6900,7 @@ bool UninterpretedOption_NamePart::MergePartialFromCodedStream( if (input->ExpectTag(16)) goto parse_is_extension; break; } - + // required bool is_extension = 2; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6611,14 +6916,15 @@ bool UninterpretedOption_NamePart::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -6637,12 +6943,12 @@ void UninterpretedOption_NamePart::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 1, this->name_part(), output); } - + // required bool is_extension = 2; if (has_is_extension()) { ::google::protobuf::internal::WireFormatLite::WriteBool(2, this->is_extension(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -6660,12 +6966,12 @@ void UninterpretedOption_NamePart::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 1, this->name_part(), target); } - + // required bool is_extension = 2; if (has_is_extension()) { target = ::google::protobuf::internal::WireFormatLite::WriteBoolToArray(2, this->is_extension(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -6675,7 +6981,7 @@ void UninterpretedOption_NamePart::SerializeWithCachedSizes( int UninterpretedOption_NamePart::ByteSize() const { int total_size = 0; - + if (_has_bits_[0 / 32] & (0xffu << (0 % 32))) { // required string name_part = 1; if (has_name_part()) { @@ -6683,12 +6989,12 @@ int UninterpretedOption_NamePart::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->name_part()); } - + // required bool is_extension = 2; if (has_is_extension()) { total_size += 1 + 1; } - + } if (!unknown_fields().empty()) { total_size += @@ -6740,7 +7046,7 @@ void UninterpretedOption_NamePart::CopyFrom(const UninterpretedOption_NamePart& bool UninterpretedOption_NamePart::IsInitialized() const { if ((_has_bits_[0] & 0x00000003) != 0x00000003) return false; - + return true; } @@ -6885,7 +7191,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(26)) goto parse_identifier_value; break; } - + // optional string identifier_value = 3; case 3: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6902,7 +7208,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(32)) goto parse_positive_int_value; break; } - + // optional uint64 positive_int_value = 4; case 4: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6918,7 +7224,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(40)) goto parse_negative_int_value; break; } - + // optional int64 negative_int_value = 5; case 5: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6934,7 +7240,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(49)) goto parse_double_value; break; } - + // optional double double_value = 6; case 6: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6950,7 +7256,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(58)) goto parse_string_value; break; } - + // optional bytes string_value = 7; case 7: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6964,7 +7270,7 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectTag(66)) goto parse_aggregate_value; break; } - + // optional string aggregate_value = 8; case 8: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -6981,14 +7287,15 @@ bool UninterpretedOption::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -7004,7 +7311,7 @@ void UninterpretedOption::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 2, this->name(i), output); } - + // optional string identifier_value = 3; if (has_identifier_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -7013,28 +7320,28 @@ void UninterpretedOption::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 3, this->identifier_value(), output); } - + // optional uint64 positive_int_value = 4; if (has_positive_int_value()) { ::google::protobuf::internal::WireFormatLite::WriteUInt64(4, this->positive_int_value(), output); } - + // optional int64 negative_int_value = 5; if (has_negative_int_value()) { ::google::protobuf::internal::WireFormatLite::WriteInt64(5, this->negative_int_value(), output); } - + // optional double double_value = 6; if (has_double_value()) { ::google::protobuf::internal::WireFormatLite::WriteDouble(6, this->double_value(), output); } - + // optional bytes string_value = 7; if (has_string_value()) { ::google::protobuf::internal::WireFormatLite::WriteBytes( 7, this->string_value(), output); } - + // optional string aggregate_value = 8; if (has_aggregate_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -7043,7 +7350,7 @@ void UninterpretedOption::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteString( 8, this->aggregate_value(), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -7058,7 +7365,7 @@ void UninterpretedOption::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 2, this->name(i), target); } - + // optional string identifier_value = 3; if (has_identifier_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -7068,29 +7375,29 @@ void UninterpretedOption::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 3, this->identifier_value(), target); } - + // optional uint64 positive_int_value = 4; if (has_positive_int_value()) { target = ::google::protobuf::internal::WireFormatLite::WriteUInt64ToArray(4, this->positive_int_value(), target); } - + // optional int64 negative_int_value = 5; if (has_negative_int_value()) { target = ::google::protobuf::internal::WireFormatLite::WriteInt64ToArray(5, this->negative_int_value(), target); } - + // optional double double_value = 6; if (has_double_value()) { target = ::google::protobuf::internal::WireFormatLite::WriteDoubleToArray(6, this->double_value(), target); } - + // optional bytes string_value = 7; if (has_string_value()) { target = ::google::protobuf::internal::WireFormatLite::WriteBytesToArray( 7, this->string_value(), target); } - + // optional string aggregate_value = 8; if (has_aggregate_value()) { ::google::protobuf::internal::WireFormat::VerifyUTF8String( @@ -7100,7 +7407,7 @@ void UninterpretedOption::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteStringToArray( 8, this->aggregate_value(), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -7110,7 +7417,7 @@ void UninterpretedOption::SerializeWithCachedSizes( int UninterpretedOption::ByteSize() const { int total_size = 0; - + if (_has_bits_[1 / 32] & (0xffu << (1 % 32))) { // optional string identifier_value = 3; if (has_identifier_value()) { @@ -7118,40 +7425,40 @@ int UninterpretedOption::ByteSize() const { ::google::protobuf::internal::WireFormatLite::StringSize( this->identifier_value()); } - + // optional uint64 positive_int_value = 4; if (has_positive_int_value()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::UInt64Size( this->positive_int_value()); } - + // optional int64 negative_int_value = 5; if (has_negative_int_value()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int64Size( this->negative_int_value()); } - + // optional double double_value = 6; if (has_double_value()) { total_size += 1 + 8; } - + // optional bytes string_value = 7; if (has_string_value()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::BytesSize( this->string_value()); } - + // optional string aggregate_value = 8; if (has_aggregate_value()) { total_size += 1 + ::google::protobuf::internal::WireFormatLite::StringSize( this->aggregate_value()); } - + } // repeated .google.protobuf.UninterpretedOption.NamePart name = 2; total_size += 1 * this->name_size(); @@ -7160,7 +7467,7 @@ int UninterpretedOption::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->name(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -7223,7 +7530,7 @@ void UninterpretedOption::CopyFrom(const UninterpretedOption& from) { } bool UninterpretedOption::IsInitialized() const { - + for (int i = 0; i < name_size(); i++) { if (!this->name(i).IsInitialized()) return false; } @@ -7259,6 +7566,8 @@ void UninterpretedOption::Swap(UninterpretedOption* other) { #ifndef _MSC_VER const int SourceCodeInfo_Location::kPathFieldNumber; const int SourceCodeInfo_Location::kSpanFieldNumber; +const int SourceCodeInfo_Location::kLeadingCommentsFieldNumber; +const int SourceCodeInfo_Location::kTrailingCommentsFieldNumber; #endif // !_MSC_VER SourceCodeInfo_Location::SourceCodeInfo_Location() @@ -7277,6 +7586,8 @@ SourceCodeInfo_Location::SourceCodeInfo_Location(const SourceCodeInfo_Location& void SourceCodeInfo_Location::SharedCtor() { _cached_size_ = 0; + leading_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + trailing_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); ::memset(_has_bits_, 0, sizeof(_has_bits_)); } @@ -7285,6 +7596,12 @@ SourceCodeInfo_Location::~SourceCodeInfo_Location() { } void SourceCodeInfo_Location::SharedDtor() { + if (leading_comments_ != &::google::protobuf::internal::kEmptyString) { + delete leading_comments_; + } + if (trailing_comments_ != &::google::protobuf::internal::kEmptyString) { + delete trailing_comments_; + } if (this != default_instance_) { } } @@ -7311,6 +7628,18 @@ SourceCodeInfo_Location* SourceCodeInfo_Location::New() const { } void SourceCodeInfo_Location::Clear() { + if (_has_bits_[2 / 32] & (0xffu << (2 % 32))) { + if (has_leading_comments()) { + if (leading_comments_ != &::google::protobuf::internal::kEmptyString) { + leading_comments_->clear(); + } + } + if (has_trailing_comments()) { + if (trailing_comments_ != &::google::protobuf::internal::kEmptyString) { + trailing_comments_->clear(); + } + } + } path_.Clear(); span_.Clear(); ::memset(_has_bits_, 0, sizeof(_has_bits_)); @@ -7342,7 +7671,7 @@ bool SourceCodeInfo_Location::MergePartialFromCodedStream( if (input->ExpectTag(18)) goto parse_span; break; } - + // repeated int32 span = 2 [packed = true]; case 2: { if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == @@ -7360,17 +7689,52 @@ bool SourceCodeInfo_Location::MergePartialFromCodedStream( } else { goto handle_uninterpreted; } + if (input->ExpectTag(26)) goto parse_leading_comments; + break; + } + + // optional string leading_comments = 3; + case 3: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED) { + parse_leading_comments: + DO_(::google::protobuf::internal::WireFormatLite::ReadString( + input, this->mutable_leading_comments())); + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->leading_comments().data(), this->leading_comments().length(), + ::google::protobuf::internal::WireFormat::PARSE); + } else { + goto handle_uninterpreted; + } + if (input->ExpectTag(34)) goto parse_trailing_comments; + break; + } + + // optional string trailing_comments = 4; + case 4: { + if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == + ::google::protobuf::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED) { + parse_trailing_comments: + DO_(::google::protobuf::internal::WireFormatLite::ReadString( + input, this->mutable_trailing_comments())); + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->trailing_comments().data(), this->trailing_comments().length(), + ::google::protobuf::internal::WireFormat::PARSE); + } else { + goto handle_uninterpreted; + } if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -7390,7 +7754,7 @@ void SourceCodeInfo_Location::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteInt32NoTag( this->path(i), output); } - + // repeated int32 span = 2 [packed = true]; if (this->span_size() > 0) { ::google::protobuf::internal::WireFormatLite::WriteTag(2, ::google::protobuf::internal::WireFormatLite::WIRETYPE_LENGTH_DELIMITED, output); @@ -7400,7 +7764,25 @@ void SourceCodeInfo_Location::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteInt32NoTag( this->span(i), output); } - + + // optional string leading_comments = 3; + if (has_leading_comments()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->leading_comments().data(), this->leading_comments().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + ::google::protobuf::internal::WireFormatLite::WriteString( + 3, this->leading_comments(), output); + } + + // optional string trailing_comments = 4; + if (has_trailing_comments()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->trailing_comments().data(), this->trailing_comments().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + ::google::protobuf::internal::WireFormatLite::WriteString( + 4, this->trailing_comments(), output); + } + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -7422,7 +7804,7 @@ void SourceCodeInfo_Location::SerializeWithCachedSizes( target = ::google::protobuf::internal::WireFormatLite:: WriteInt32NoTagToArray(this->path(i), target); } - + // repeated int32 span = 2 [packed = true]; if (this->span_size() > 0) { target = ::google::protobuf::internal::WireFormatLite::WriteTagToArray( @@ -7436,7 +7818,27 @@ void SourceCodeInfo_Location::SerializeWithCachedSizes( target = ::google::protobuf::internal::WireFormatLite:: WriteInt32NoTagToArray(this->span(i), target); } - + + // optional string leading_comments = 3; + if (has_leading_comments()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->leading_comments().data(), this->leading_comments().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + target = + ::google::protobuf::internal::WireFormatLite::WriteStringToArray( + 3, this->leading_comments(), target); + } + + // optional string trailing_comments = 4; + if (has_trailing_comments()) { + ::google::protobuf::internal::WireFormat::VerifyUTF8String( + this->trailing_comments().data(), this->trailing_comments().length(), + ::google::protobuf::internal::WireFormat::SERIALIZE); + target = + ::google::protobuf::internal::WireFormatLite::WriteStringToArray( + 4, this->trailing_comments(), target); + } + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -7446,7 +7848,23 @@ void SourceCodeInfo_Location::SerializeWithCachedSizes( int SourceCodeInfo_Location::ByteSize() const { int total_size = 0; - + + if (_has_bits_[2 / 32] & (0xffu << (2 % 32))) { + // optional string leading_comments = 3; + if (has_leading_comments()) { + total_size += 1 + + ::google::protobuf::internal::WireFormatLite::StringSize( + this->leading_comments()); + } + + // optional string trailing_comments = 4; + if (has_trailing_comments()) { + total_size += 1 + + ::google::protobuf::internal::WireFormatLite::StringSize( + this->trailing_comments()); + } + + } // repeated int32 path = 1 [packed = true]; { int data_size = 0; @@ -7458,10 +7876,12 @@ int SourceCodeInfo_Location::ByteSize() const { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int32Size(data_size); } + GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN(); _path_cached_byte_size_ = data_size; + GOOGLE_SAFE_CONCURRENT_WRITES_END(); total_size += data_size; } - + // repeated int32 span = 2 [packed = true]; { int data_size = 0; @@ -7473,10 +7893,12 @@ int SourceCodeInfo_Location::ByteSize() const { total_size += 1 + ::google::protobuf::internal::WireFormatLite::Int32Size(data_size); } + GOOGLE_SAFE_CONCURRENT_WRITES_BEGIN(); _span_cached_byte_size_ = data_size; + GOOGLE_SAFE_CONCURRENT_WRITES_END(); total_size += data_size; } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -7504,6 +7926,14 @@ void SourceCodeInfo_Location::MergeFrom(const SourceCodeInfo_Location& from) { GOOGLE_CHECK_NE(&from, this); path_.MergeFrom(from.path_); span_.MergeFrom(from.span_); + if (from._has_bits_[2 / 32] & (0xffu << (2 % 32))) { + if (from.has_leading_comments()) { + set_leading_comments(from.leading_comments()); + } + if (from.has_trailing_comments()) { + set_trailing_comments(from.trailing_comments()); + } + } mutable_unknown_fields()->MergeFrom(from.unknown_fields()); } @@ -7520,7 +7950,7 @@ void SourceCodeInfo_Location::CopyFrom(const SourceCodeInfo_Location& from) { } bool SourceCodeInfo_Location::IsInitialized() const { - + return true; } @@ -7528,6 +7958,8 @@ void SourceCodeInfo_Location::Swap(SourceCodeInfo_Location* other) { if (other != this) { path_.Swap(&other->path_); span_.Swap(&other->span_); + std::swap(leading_comments_, other->leading_comments_); + std::swap(trailing_comments_, other->trailing_comments_); std::swap(_has_bits_[0], other->_has_bits_[0]); _unknown_fields_.Swap(&other->_unknown_fields_); std::swap(_cached_size_, other->_cached_size_); @@ -7624,14 +8056,15 @@ bool SourceCodeInfo::MergePartialFromCodedStream( if (input->ExpectAtEnd()) return true; break; } - + default: { handle_uninterpreted: if (::google::protobuf::internal::WireFormatLite::GetTagWireType(tag) == ::google::protobuf::internal::WireFormatLite::WIRETYPE_END_GROUP) { return true; } - DO_(::google::protobuf::internal::WireFormatLite::SkipField(input, tag, mutable_unknown_fields())); + DO_(::google::protobuf::internal::WireFormatLite::SkipField( + input, tag, mutable_unknown_fields())); break; } } @@ -7647,7 +8080,7 @@ void SourceCodeInfo::SerializeWithCachedSizes( ::google::protobuf::internal::WireFormatLite::WriteMessageMaybeToArray( 1, this->location(i), output); } - + if (!unknown_fields().empty()) { ::google::protobuf::internal::WireFormatLite::SerializeUnknownFields( unknown_fields(), output); @@ -7662,7 +8095,7 @@ void SourceCodeInfo::SerializeWithCachedSizes( WriteMessageNoVirtualToArray( 1, this->location(i), target); } - + if (!unknown_fields().empty()) { target = ::google::protobuf::internal::WireFormatLite::SerializeUnknownFieldsToArray( unknown_fields(), target); @@ -7672,7 +8105,7 @@ void SourceCodeInfo::SerializeWithCachedSizes( int SourceCodeInfo::ByteSize() const { int total_size = 0; - + // repeated .google.protobuf.SourceCodeInfo.Location location = 1; total_size += 1 * this->location_size(); for (int i = 0; i < this->location_size(); i++) { @@ -7680,7 +8113,7 @@ int SourceCodeInfo::ByteSize() const { ::google::protobuf::internal::WireFormatLite::MessageSizeNoVirtual( this->location(i)); } - + if (!unknown_fields().empty()) { total_size += ::google::protobuf::internal::WireFormatLite::ComputeUnknownFieldsSize( @@ -7723,7 +8156,7 @@ void SourceCodeInfo::CopyFrom(const SourceCodeInfo& from) { } bool SourceCodeInfo::IsInitialized() const { - + return true; } diff --git a/third_party/protobuf/src/google/protobuf/descriptor.pb.h b/third_party/protobuf/src/google/protobuf/descriptor.pb.h index 1f91d79..5c1175c 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor.pb.h +++ b/third_party/protobuf/src/google/protobuf/descriptor.pb.h @@ -20,10 +20,11 @@ #endif #include <google/protobuf/generated_message_util.h> +#include <google/protobuf/message.h> #include <google/protobuf/repeated_field.h> #include <google/protobuf/extension_set.h> +#include <google/protobuf/generated_enum_reflection.h> #include <google/protobuf/unknown_field_set.h> -#include <google/protobuf/generated_message_reflection.h> // @@protoc_insertion_point(includes) namespace google { @@ -156,29 +157,29 @@ class LIBPROTOBUF_EXPORT FileDescriptorSet : public ::google::protobuf::Message public: FileDescriptorSet(); virtual ~FileDescriptorSet(); - + FileDescriptorSet(const FileDescriptorSet& from); - + inline FileDescriptorSet& operator=(const FileDescriptorSet& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const FileDescriptorSet& default_instance(); - + void Swap(FileDescriptorSet* other); - + // implements Message ---------------------------------------------- - + FileDescriptorSet* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -186,7 +187,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorSet : public ::google::protobuf::Message void MergeFrom(const FileDescriptorSet& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -199,13 +200,13 @@ class LIBPROTOBUF_EXPORT FileDescriptorSet : public ::google::protobuf::Message void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.FileDescriptorProto file = 1; inline int file_size() const; inline void clear_file(); @@ -217,21 +218,21 @@ class LIBPROTOBUF_EXPORT FileDescriptorSet : public ::google::protobuf::Message file() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::FileDescriptorProto >* mutable_file(); - + // @@protoc_insertion_point(class_scope:google.protobuf.FileDescriptorSet) private: - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::FileDescriptorProto > file_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static FileDescriptorSet* default_instance_; }; @@ -241,29 +242,29 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag public: FileDescriptorProto(); virtual ~FileDescriptorProto(); - + FileDescriptorProto(const FileDescriptorProto& from); - + inline FileDescriptorProto& operator=(const FileDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const FileDescriptorProto& default_instance(); - + void Swap(FileDescriptorProto* other); - + // implements Message ---------------------------------------------- - + FileDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -271,7 +272,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag void MergeFrom(const FileDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -284,13 +285,13 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -301,7 +302,8 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // optional string package = 2; inline bool has_package() const; inline void clear_package(); @@ -312,7 +314,8 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline void set_package(const char* value, size_t size); inline ::std::string* mutable_package(); inline ::std::string* release_package(); - + inline void set_allocated_package(::std::string* package); + // repeated string dependency = 3; inline int dependency_size() const; inline void clear_dependency(); @@ -328,7 +331,31 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline void add_dependency(const char* value, size_t size); inline const ::google::protobuf::RepeatedPtrField< ::std::string>& dependency() const; inline ::google::protobuf::RepeatedPtrField< ::std::string>* mutable_dependency(); - + + // repeated int32 public_dependency = 10; + inline int public_dependency_size() const; + inline void clear_public_dependency(); + static const int kPublicDependencyFieldNumber = 10; + inline ::google::protobuf::int32 public_dependency(int index) const; + inline void set_public_dependency(int index, ::google::protobuf::int32 value); + inline void add_public_dependency(::google::protobuf::int32 value); + inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + public_dependency() const; + inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_public_dependency(); + + // repeated int32 weak_dependency = 11; + inline int weak_dependency_size() const; + inline void clear_weak_dependency(); + static const int kWeakDependencyFieldNumber = 11; + inline ::google::protobuf::int32 weak_dependency(int index) const; + inline void set_weak_dependency(int index, ::google::protobuf::int32 value); + inline void add_weak_dependency(::google::protobuf::int32 value); + inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& + weak_dependency() const; + inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* + mutable_weak_dependency(); + // repeated .google.protobuf.DescriptorProto message_type = 4; inline int message_type_size() const; inline void clear_message_type(); @@ -340,7 +367,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag message_type() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::DescriptorProto >* mutable_message_type(); - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 5; inline int enum_type_size() const; inline void clear_enum_type(); @@ -352,7 +379,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag enum_type() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumDescriptorProto >* mutable_enum_type(); - + // repeated .google.protobuf.ServiceDescriptorProto service = 6; inline int service_size() const; inline void clear_service(); @@ -364,7 +391,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag service() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::ServiceDescriptorProto >* mutable_service(); - + // repeated .google.protobuf.FieldDescriptorProto extension = 7; inline int extension_size() const; inline void clear_extension(); @@ -376,7 +403,7 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag extension() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto >* mutable_extension(); - + // optional .google.protobuf.FileOptions options = 8; inline bool has_options() const; inline void clear_options(); @@ -384,7 +411,8 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline const ::google::protobuf::FileOptions& options() const; inline ::google::protobuf::FileOptions* mutable_options(); inline ::google::protobuf::FileOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::FileOptions* options); + // optional .google.protobuf.SourceCodeInfo source_code_info = 9; inline bool has_source_code_info() const; inline void clear_source_code_info(); @@ -392,7 +420,8 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline const ::google::protobuf::SourceCodeInfo& source_code_info() const; inline ::google::protobuf::SourceCodeInfo* mutable_source_code_info(); inline ::google::protobuf::SourceCodeInfo* release_source_code_info(); - + inline void set_allocated_source_code_info(::google::protobuf::SourceCodeInfo* source_code_info); + // @@protoc_insertion_point(class_scope:google.protobuf.FileDescriptorProto) private: inline void set_has_name(); @@ -403,26 +432,28 @@ class LIBPROTOBUF_EXPORT FileDescriptorProto : public ::google::protobuf::Messag inline void clear_has_options(); inline void set_has_source_code_info(); inline void clear_has_source_code_info(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::std::string* package_; ::google::protobuf::RepeatedPtrField< ::std::string> dependency_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > public_dependency_; + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > weak_dependency_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::DescriptorProto > message_type_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumDescriptorProto > enum_type_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::ServiceDescriptorProto > service_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto > extension_; ::google::protobuf::FileOptions* options_; ::google::protobuf::SourceCodeInfo* source_code_info_; - + mutable int _cached_size_; - ::google::protobuf::uint32 _has_bits_[(9 + 31) / 32]; - + ::google::protobuf::uint32 _has_bits_[(11 + 31) / 32]; + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static FileDescriptorProto* default_instance_; }; @@ -432,29 +463,29 @@ class LIBPROTOBUF_EXPORT DescriptorProto_ExtensionRange : public ::google::proto public: DescriptorProto_ExtensionRange(); virtual ~DescriptorProto_ExtensionRange(); - + DescriptorProto_ExtensionRange(const DescriptorProto_ExtensionRange& from); - + inline DescriptorProto_ExtensionRange& operator=(const DescriptorProto_ExtensionRange& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const DescriptorProto_ExtensionRange& default_instance(); - + void Swap(DescriptorProto_ExtensionRange* other); - + // implements Message ---------------------------------------------- - + DescriptorProto_ExtensionRange* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -462,7 +493,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto_ExtensionRange : public ::google::proto void MergeFrom(const DescriptorProto_ExtensionRange& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -475,46 +506,46 @@ class LIBPROTOBUF_EXPORT DescriptorProto_ExtensionRange : public ::google::proto void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional int32 start = 1; inline bool has_start() const; inline void clear_start(); static const int kStartFieldNumber = 1; inline ::google::protobuf::int32 start() const; inline void set_start(::google::protobuf::int32 value); - + // optional int32 end = 2; inline bool has_end() const; inline void clear_end(); static const int kEndFieldNumber = 2; inline ::google::protobuf::int32 end() const; inline void set_end(::google::protobuf::int32 value); - + // @@protoc_insertion_point(class_scope:google.protobuf.DescriptorProto.ExtensionRange) private: inline void set_has_start(); inline void clear_has_start(); inline void set_has_end(); inline void clear_has_end(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::int32 start_; ::google::protobuf::int32 end_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(2 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static DescriptorProto_ExtensionRange* default_instance_; }; @@ -524,29 +555,29 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { public: DescriptorProto(); virtual ~DescriptorProto(); - + DescriptorProto(const DescriptorProto& from); - + inline DescriptorProto& operator=(const DescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const DescriptorProto& default_instance(); - + void Swap(DescriptorProto* other); - + // implements Message ---------------------------------------------- - + DescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -554,7 +585,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { void MergeFrom(const DescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -567,15 +598,15 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef DescriptorProto_ExtensionRange ExtensionRange; - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -586,7 +617,8 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // repeated .google.protobuf.FieldDescriptorProto field = 2; inline int field_size() const; inline void clear_field(); @@ -598,7 +630,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { field() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto >* mutable_field(); - + // repeated .google.protobuf.FieldDescriptorProto extension = 6; inline int extension_size() const; inline void clear_extension(); @@ -610,7 +642,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { extension() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto >* mutable_extension(); - + // repeated .google.protobuf.DescriptorProto nested_type = 3; inline int nested_type_size() const; inline void clear_nested_type(); @@ -622,7 +654,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { nested_type() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::DescriptorProto >* mutable_nested_type(); - + // repeated .google.protobuf.EnumDescriptorProto enum_type = 4; inline int enum_type_size() const; inline void clear_enum_type(); @@ -634,7 +666,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { enum_type() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumDescriptorProto >* mutable_enum_type(); - + // repeated .google.protobuf.DescriptorProto.ExtensionRange extension_range = 5; inline int extension_range_size() const; inline void clear_extension_range(); @@ -646,7 +678,7 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { extension_range() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::DescriptorProto_ExtensionRange >* mutable_extension_range(); - + // optional .google.protobuf.MessageOptions options = 7; inline bool has_options() const; inline void clear_options(); @@ -654,16 +686,17 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { inline const ::google::protobuf::MessageOptions& options() const; inline ::google::protobuf::MessageOptions* mutable_options(); inline ::google::protobuf::MessageOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::MessageOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.DescriptorProto) private: inline void set_has_name(); inline void clear_has_name(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto > field_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::FieldDescriptorProto > extension_; @@ -671,14 +704,14 @@ class LIBPROTOBUF_EXPORT DescriptorProto : public ::google::protobuf::Message { ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumDescriptorProto > enum_type_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::DescriptorProto_ExtensionRange > extension_range_; ::google::protobuf::MessageOptions* options_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(7 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static DescriptorProto* default_instance_; }; @@ -688,29 +721,29 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa public: FieldDescriptorProto(); virtual ~FieldDescriptorProto(); - + FieldDescriptorProto(const FieldDescriptorProto& from); - + inline FieldDescriptorProto& operator=(const FieldDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const FieldDescriptorProto& default_instance(); - + void Swap(FieldDescriptorProto* other); - + // implements Message ---------------------------------------------- - + FieldDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -718,7 +751,7 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa void MergeFrom(const FieldDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -731,11 +764,11 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef FieldDescriptorProto_Type Type; static const Type TYPE_DOUBLE = FieldDescriptorProto_Type_TYPE_DOUBLE; static const Type TYPE_FLOAT = FieldDescriptorProto_Type_TYPE_FLOAT; @@ -775,7 +808,7 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa Type* value) { return FieldDescriptorProto_Type_Parse(name, value); } - + typedef FieldDescriptorProto_Label Label; static const Label LABEL_OPTIONAL = FieldDescriptorProto_Label_LABEL_OPTIONAL; static const Label LABEL_REQUIRED = FieldDescriptorProto_Label_LABEL_REQUIRED; @@ -800,9 +833,9 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa Label* value) { return FieldDescriptorProto_Label_Parse(name, value); } - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -813,28 +846,29 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // optional int32 number = 3; inline bool has_number() const; inline void clear_number(); static const int kNumberFieldNumber = 3; inline ::google::protobuf::int32 number() const; inline void set_number(::google::protobuf::int32 value); - + // optional .google.protobuf.FieldDescriptorProto.Label label = 4; inline bool has_label() const; inline void clear_label(); static const int kLabelFieldNumber = 4; inline ::google::protobuf::FieldDescriptorProto_Label label() const; inline void set_label(::google::protobuf::FieldDescriptorProto_Label value); - + // optional .google.protobuf.FieldDescriptorProto.Type type = 5; inline bool has_type() const; inline void clear_type(); static const int kTypeFieldNumber = 5; inline ::google::protobuf::FieldDescriptorProto_Type type() const; inline void set_type(::google::protobuf::FieldDescriptorProto_Type value); - + // optional string type_name = 6; inline bool has_type_name() const; inline void clear_type_name(); @@ -845,7 +879,8 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline void set_type_name(const char* value, size_t size); inline ::std::string* mutable_type_name(); inline ::std::string* release_type_name(); - + inline void set_allocated_type_name(::std::string* type_name); + // optional string extendee = 2; inline bool has_extendee() const; inline void clear_extendee(); @@ -856,7 +891,8 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline void set_extendee(const char* value, size_t size); inline ::std::string* mutable_extendee(); inline ::std::string* release_extendee(); - + inline void set_allocated_extendee(::std::string* extendee); + // optional string default_value = 7; inline bool has_default_value() const; inline void clear_default_value(); @@ -867,7 +903,8 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline void set_default_value(const char* value, size_t size); inline ::std::string* mutable_default_value(); inline ::std::string* release_default_value(); - + inline void set_allocated_default_value(::std::string* default_value); + // optional .google.protobuf.FieldOptions options = 8; inline bool has_options() const; inline void clear_options(); @@ -875,7 +912,8 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline const ::google::protobuf::FieldOptions& options() const; inline ::google::protobuf::FieldOptions* mutable_options(); inline ::google::protobuf::FieldOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::FieldOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.FieldDescriptorProto) private: inline void set_has_name(); @@ -894,9 +932,9 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa inline void clear_has_default_value(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::google::protobuf::int32 number_; int label_; @@ -905,14 +943,14 @@ class LIBPROTOBUF_EXPORT FieldDescriptorProto : public ::google::protobuf::Messa ::std::string* default_value_; ::google::protobuf::FieldOptions* options_; int type_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(8 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static FieldDescriptorProto* default_instance_; }; @@ -922,29 +960,29 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag public: EnumDescriptorProto(); virtual ~EnumDescriptorProto(); - + EnumDescriptorProto(const EnumDescriptorProto& from); - + inline EnumDescriptorProto& operator=(const EnumDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const EnumDescriptorProto& default_instance(); - + void Swap(EnumDescriptorProto* other); - + // implements Message ---------------------------------------------- - + EnumDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -952,7 +990,7 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag void MergeFrom(const EnumDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -965,13 +1003,13 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -982,7 +1020,8 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // repeated .google.protobuf.EnumValueDescriptorProto value = 2; inline int value_size() const; inline void clear_value(); @@ -994,7 +1033,7 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag value() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumValueDescriptorProto >* mutable_value(); - + // optional .google.protobuf.EnumOptions options = 3; inline bool has_options() const; inline void clear_options(); @@ -1002,27 +1041,28 @@ class LIBPROTOBUF_EXPORT EnumDescriptorProto : public ::google::protobuf::Messag inline const ::google::protobuf::EnumOptions& options() const; inline ::google::protobuf::EnumOptions* mutable_options(); inline ::google::protobuf::EnumOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::EnumOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.EnumDescriptorProto) private: inline void set_has_name(); inline void clear_has_name(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::EnumValueDescriptorProto > value_; ::google::protobuf::EnumOptions* options_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static EnumDescriptorProto* default_instance_; }; @@ -1032,29 +1072,29 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M public: EnumValueDescriptorProto(); virtual ~EnumValueDescriptorProto(); - + EnumValueDescriptorProto(const EnumValueDescriptorProto& from); - + inline EnumValueDescriptorProto& operator=(const EnumValueDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const EnumValueDescriptorProto& default_instance(); - + void Swap(EnumValueDescriptorProto* other); - + // implements Message ---------------------------------------------- - + EnumValueDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1062,7 +1102,7 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M void MergeFrom(const EnumValueDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1075,13 +1115,13 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -1092,14 +1132,15 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // optional int32 number = 2; inline bool has_number() const; inline void clear_number(); static const int kNumberFieldNumber = 2; inline ::google::protobuf::int32 number() const; inline void set_number(::google::protobuf::int32 value); - + // optional .google.protobuf.EnumValueOptions options = 3; inline bool has_options() const; inline void clear_options(); @@ -1107,7 +1148,8 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M inline const ::google::protobuf::EnumValueOptions& options() const; inline ::google::protobuf::EnumValueOptions* mutable_options(); inline ::google::protobuf::EnumValueOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::EnumValueOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.EnumValueDescriptorProto) private: inline void set_has_name(); @@ -1116,20 +1158,20 @@ class LIBPROTOBUF_EXPORT EnumValueDescriptorProto : public ::google::protobuf::M inline void clear_has_number(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::google::protobuf::EnumValueOptions* options_; ::google::protobuf::int32 number_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static EnumValueDescriptorProto* default_instance_; }; @@ -1139,29 +1181,29 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes public: ServiceDescriptorProto(); virtual ~ServiceDescriptorProto(); - + ServiceDescriptorProto(const ServiceDescriptorProto& from); - + inline ServiceDescriptorProto& operator=(const ServiceDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const ServiceDescriptorProto& default_instance(); - + void Swap(ServiceDescriptorProto* other); - + // implements Message ---------------------------------------------- - + ServiceDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1169,7 +1211,7 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes void MergeFrom(const ServiceDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1182,13 +1224,13 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -1199,7 +1241,8 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // repeated .google.protobuf.MethodDescriptorProto method = 2; inline int method_size() const; inline void clear_method(); @@ -1211,7 +1254,7 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes method() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::MethodDescriptorProto >* mutable_method(); - + // optional .google.protobuf.ServiceOptions options = 3; inline bool has_options() const; inline void clear_options(); @@ -1219,27 +1262,28 @@ class LIBPROTOBUF_EXPORT ServiceDescriptorProto : public ::google::protobuf::Mes inline const ::google::protobuf::ServiceOptions& options() const; inline ::google::protobuf::ServiceOptions* mutable_options(); inline ::google::protobuf::ServiceOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::ServiceOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.ServiceDescriptorProto) private: inline void set_has_name(); inline void clear_has_name(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::MethodDescriptorProto > method_; ::google::protobuf::ServiceOptions* options_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static ServiceDescriptorProto* default_instance_; }; @@ -1249,29 +1293,29 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess public: MethodDescriptorProto(); virtual ~MethodDescriptorProto(); - + MethodDescriptorProto(const MethodDescriptorProto& from); - + inline MethodDescriptorProto& operator=(const MethodDescriptorProto& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const MethodDescriptorProto& default_instance(); - + void Swap(MethodDescriptorProto* other); - + // implements Message ---------------------------------------------- - + MethodDescriptorProto* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1279,7 +1323,7 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess void MergeFrom(const MethodDescriptorProto& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1292,13 +1336,13 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional string name = 1; inline bool has_name() const; inline void clear_name(); @@ -1309,7 +1353,8 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess inline void set_name(const char* value, size_t size); inline ::std::string* mutable_name(); inline ::std::string* release_name(); - + inline void set_allocated_name(::std::string* name); + // optional string input_type = 2; inline bool has_input_type() const; inline void clear_input_type(); @@ -1320,7 +1365,8 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess inline void set_input_type(const char* value, size_t size); inline ::std::string* mutable_input_type(); inline ::std::string* release_input_type(); - + inline void set_allocated_input_type(::std::string* input_type); + // optional string output_type = 3; inline bool has_output_type() const; inline void clear_output_type(); @@ -1331,7 +1377,8 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess inline void set_output_type(const char* value, size_t size); inline ::std::string* mutable_output_type(); inline ::std::string* release_output_type(); - + inline void set_allocated_output_type(::std::string* output_type); + // optional .google.protobuf.MethodOptions options = 4; inline bool has_options() const; inline void clear_options(); @@ -1339,7 +1386,8 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess inline const ::google::protobuf::MethodOptions& options() const; inline ::google::protobuf::MethodOptions* mutable_options(); inline ::google::protobuf::MethodOptions* release_options(); - + inline void set_allocated_options(::google::protobuf::MethodOptions* options); + // @@protoc_insertion_point(class_scope:google.protobuf.MethodDescriptorProto) private: inline void set_has_name(); @@ -1350,21 +1398,21 @@ class LIBPROTOBUF_EXPORT MethodDescriptorProto : public ::google::protobuf::Mess inline void clear_has_output_type(); inline void set_has_options(); inline void clear_has_options(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_; ::std::string* input_type_; ::std::string* output_type_; ::google::protobuf::MethodOptions* options_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(4 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static MethodDescriptorProto* default_instance_; }; @@ -1374,29 +1422,29 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { public: FileOptions(); virtual ~FileOptions(); - + FileOptions(const FileOptions& from); - + inline FileOptions& operator=(const FileOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const FileOptions& default_instance(); - + void Swap(FileOptions* other); - + // implements Message ---------------------------------------------- - + FileOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1404,7 +1452,7 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { void MergeFrom(const FileOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1417,11 +1465,11 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef FileOptions_OptimizeMode OptimizeMode; static const OptimizeMode SPEED = FileOptions_OptimizeMode_SPEED; static const OptimizeMode CODE_SIZE = FileOptions_OptimizeMode_CODE_SIZE; @@ -1446,9 +1494,9 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { OptimizeMode* value) { return FileOptions_OptimizeMode_Parse(name, value); } - + // accessors ------------------------------------------------------- - + // optional string java_package = 1; inline bool has_java_package() const; inline void clear_java_package(); @@ -1459,7 +1507,8 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { inline void set_java_package(const char* value, size_t size); inline ::std::string* mutable_java_package(); inline ::std::string* release_java_package(); - + inline void set_allocated_java_package(::std::string* java_package); + // optional string java_outer_classname = 8; inline bool has_java_outer_classname() const; inline void clear_java_outer_classname(); @@ -1470,56 +1519,69 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { inline void set_java_outer_classname(const char* value, size_t size); inline ::std::string* mutable_java_outer_classname(); inline ::std::string* release_java_outer_classname(); - + inline void set_allocated_java_outer_classname(::std::string* java_outer_classname); + // optional bool java_multiple_files = 10 [default = false]; inline bool has_java_multiple_files() const; inline void clear_java_multiple_files(); static const int kJavaMultipleFilesFieldNumber = 10; inline bool java_multiple_files() const; inline void set_java_multiple_files(bool value); - - // optional bool retain_unknown_fields = 11 [default = false]; + + // optional bool retain_unknown_fields = 12 [default = false]; inline bool has_retain_unknown_fields() const; inline void clear_retain_unknown_fields(); - static const int kRetainUnknownFieldsFieldNumber = 11; + static const int kRetainUnknownFieldsFieldNumber = 12; inline bool retain_unknown_fields() const; inline void set_retain_unknown_fields(bool value); - + // optional bool java_generate_equals_and_hash = 20 [default = false]; inline bool has_java_generate_equals_and_hash() const; inline void clear_java_generate_equals_and_hash(); static const int kJavaGenerateEqualsAndHashFieldNumber = 20; inline bool java_generate_equals_and_hash() const; inline void set_java_generate_equals_and_hash(bool value); - + // optional .google.protobuf.FileOptions.OptimizeMode optimize_for = 9 [default = SPEED]; inline bool has_optimize_for() const; inline void clear_optimize_for(); static const int kOptimizeForFieldNumber = 9; inline ::google::protobuf::FileOptions_OptimizeMode optimize_for() const; inline void set_optimize_for(::google::protobuf::FileOptions_OptimizeMode value); - + + // optional string go_package = 11; + inline bool has_go_package() const; + inline void clear_go_package(); + static const int kGoPackageFieldNumber = 11; + inline const ::std::string& go_package() const; + inline void set_go_package(const ::std::string& value); + inline void set_go_package(const char* value); + inline void set_go_package(const char* value, size_t size); + inline ::std::string* mutable_go_package(); + inline ::std::string* release_go_package(); + inline void set_allocated_go_package(::std::string* go_package); + // optional bool cc_generic_services = 16 [default = false]; inline bool has_cc_generic_services() const; inline void clear_cc_generic_services(); static const int kCcGenericServicesFieldNumber = 16; inline bool cc_generic_services() const; inline void set_cc_generic_services(bool value); - + // optional bool java_generic_services = 17 [default = false]; inline bool has_java_generic_services() const; inline void clear_java_generic_services(); static const int kJavaGenericServicesFieldNumber = 17; inline bool java_generic_services() const; inline void set_java_generic_services(bool value); - + // optional bool py_generic_services = 18 [default = false]; inline bool has_py_generic_services() const; inline void clear_py_generic_services(); static const int kPyGenericServicesFieldNumber = 18; inline bool py_generic_services() const; inline void set_py_generic_services(bool value); - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -1531,7 +1593,7 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(FileOptions) // @@protoc_insertion_point(class_scope:google.protobuf.FileOptions) private: @@ -1547,17 +1609,19 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { inline void clear_has_java_generate_equals_and_hash(); inline void set_has_optimize_for(); inline void clear_has_optimize_for(); + inline void set_has_go_package(); + inline void clear_has_go_package(); inline void set_has_cc_generic_services(); inline void clear_has_cc_generic_services(); inline void set_has_java_generic_services(); inline void clear_has_java_generic_services(); inline void set_has_py_generic_services(); inline void clear_has_py_generic_services(); - + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* java_package_; ::std::string* java_outer_classname_; bool java_multiple_files_; @@ -1565,17 +1629,18 @@ class LIBPROTOBUF_EXPORT FileOptions : public ::google::protobuf::Message { bool java_generate_equals_and_hash_; bool cc_generic_services_; int optimize_for_; + ::std::string* go_package_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; bool java_generic_services_; bool py_generic_services_; - + mutable int _cached_size_; - ::google::protobuf::uint32 _has_bits_[(10 + 31) / 32]; - + ::google::protobuf::uint32 _has_bits_[(11 + 31) / 32]; + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static FileOptions* default_instance_; }; @@ -1585,29 +1650,29 @@ class LIBPROTOBUF_EXPORT MessageOptions : public ::google::protobuf::Message { public: MessageOptions(); virtual ~MessageOptions(); - + MessageOptions(const MessageOptions& from); - + inline MessageOptions& operator=(const MessageOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const MessageOptions& default_instance(); - + void Swap(MessageOptions* other); - + // implements Message ---------------------------------------------- - + MessageOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1615,7 +1680,7 @@ class LIBPROTOBUF_EXPORT MessageOptions : public ::google::protobuf::Message { void MergeFrom(const MessageOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1628,27 +1693,27 @@ class LIBPROTOBUF_EXPORT MessageOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // optional bool message_set_wire_format = 1 [default = false]; inline bool has_message_set_wire_format() const; inline void clear_message_set_wire_format(); static const int kMessageSetWireFormatFieldNumber = 1; inline bool message_set_wire_format() const; inline void set_message_set_wire_format(bool value); - + // optional bool no_standard_descriptor_accessor = 2 [default = false]; inline bool has_no_standard_descriptor_accessor() const; inline void clear_no_standard_descriptor_accessor(); static const int kNoStandardDescriptorAccessorFieldNumber = 2; inline bool no_standard_descriptor_accessor() const; inline void set_no_standard_descriptor_accessor(bool value); - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -1660,7 +1725,7 @@ class LIBPROTOBUF_EXPORT MessageOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(MessageOptions) // @@protoc_insertion_point(class_scope:google.protobuf.MessageOptions) private: @@ -1668,22 +1733,22 @@ class LIBPROTOBUF_EXPORT MessageOptions : public ::google::protobuf::Message { inline void clear_has_message_set_wire_format(); inline void set_has_no_standard_descriptor_accessor(); inline void clear_has_no_standard_descriptor_accessor(); - + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; bool message_set_wire_format_; bool no_standard_descriptor_accessor_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(3 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static MessageOptions* default_instance_; }; @@ -1693,29 +1758,29 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { public: FieldOptions(); virtual ~FieldOptions(); - + FieldOptions(const FieldOptions& from); - + inline FieldOptions& operator=(const FieldOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const FieldOptions& default_instance(); - + void Swap(FieldOptions* other); - + // implements Message ---------------------------------------------- - + FieldOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1723,7 +1788,7 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { void MergeFrom(const FieldOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1736,11 +1801,11 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef FieldOptions_CType CType; static const CType STRING = FieldOptions_CType_STRING; static const CType CORD = FieldOptions_CType_CORD; @@ -1765,30 +1830,37 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { CType* value) { return FieldOptions_CType_Parse(name, value); } - + // accessors ------------------------------------------------------- - + // optional .google.protobuf.FieldOptions.CType ctype = 1 [default = STRING]; inline bool has_ctype() const; inline void clear_ctype(); static const int kCtypeFieldNumber = 1; inline ::google::protobuf::FieldOptions_CType ctype() const; inline void set_ctype(::google::protobuf::FieldOptions_CType value); - + // optional bool packed = 2; inline bool has_packed() const; inline void clear_packed(); static const int kPackedFieldNumber = 2; inline bool packed() const; inline void set_packed(bool value); - + + // optional bool lazy = 5 [default = false]; + inline bool has_lazy() const; + inline void clear_lazy(); + static const int kLazyFieldNumber = 5; + inline bool lazy() const; + inline void set_lazy(bool value); + // optional bool deprecated = 3 [default = false]; inline bool has_deprecated() const; inline void clear_deprecated(); static const int kDeprecatedFieldNumber = 3; inline bool deprecated() const; inline void set_deprecated(bool value); - + // optional string experimental_map_key = 9; inline bool has_experimental_map_key() const; inline void clear_experimental_map_key(); @@ -1799,7 +1871,15 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { inline void set_experimental_map_key(const char* value, size_t size); inline ::std::string* mutable_experimental_map_key(); inline ::std::string* release_experimental_map_key(); - + inline void set_allocated_experimental_map_key(::std::string* experimental_map_key); + + // optional bool weak = 10 [default = false]; + inline bool has_weak() const; + inline void clear_weak(); + static const int kWeakFieldNumber = 10; + inline bool weak() const; + inline void set_weak(bool value); + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -1811,7 +1891,7 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(FieldOptions) // @@protoc_insertion_point(class_scope:google.protobuf.FieldOptions) private: @@ -1819,28 +1899,34 @@ class LIBPROTOBUF_EXPORT FieldOptions : public ::google::protobuf::Message { inline void clear_has_ctype(); inline void set_has_packed(); inline void clear_has_packed(); + inline void set_has_lazy(); + inline void clear_has_lazy(); inline void set_has_deprecated(); inline void clear_has_deprecated(); inline void set_has_experimental_map_key(); inline void clear_has_experimental_map_key(); - + inline void set_has_weak(); + inline void clear_has_weak(); + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + int ctype_; bool packed_; + bool lazy_; bool deprecated_; + bool weak_; ::std::string* experimental_map_key_; ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; - + mutable int _cached_size_; - ::google::protobuf::uint32 _has_bits_[(5 + 31) / 32]; - + ::google::protobuf::uint32 _has_bits_[(7 + 31) / 32]; + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static FieldOptions* default_instance_; }; @@ -1850,29 +1936,29 @@ class LIBPROTOBUF_EXPORT EnumOptions : public ::google::protobuf::Message { public: EnumOptions(); virtual ~EnumOptions(); - + EnumOptions(const EnumOptions& from); - + inline EnumOptions& operator=(const EnumOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const EnumOptions& default_instance(); - + void Swap(EnumOptions* other); - + // implements Message ---------------------------------------------- - + EnumOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1880,7 +1966,7 @@ class LIBPROTOBUF_EXPORT EnumOptions : public ::google::protobuf::Message { void MergeFrom(const EnumOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1893,13 +1979,20 @@ class LIBPROTOBUF_EXPORT EnumOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + + // optional bool allow_alias = 2 [default = true]; + inline bool has_allow_alias() const; + inline void clear_allow_alias(); + static const int kAllowAliasFieldNumber = 2; + inline bool allow_alias() const; + inline void set_allow_alias(bool value); + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -1911,24 +2004,27 @@ class LIBPROTOBUF_EXPORT EnumOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(EnumOptions) // @@protoc_insertion_point(class_scope:google.protobuf.EnumOptions) private: - + inline void set_has_allow_alias(); + inline void clear_has_allow_alias(); + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; - + bool allow_alias_; + mutable int _cached_size_; - ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + ::google::protobuf::uint32 _has_bits_[(2 + 31) / 32]; + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static EnumOptions* default_instance_; }; @@ -1938,29 +2034,29 @@ class LIBPROTOBUF_EXPORT EnumValueOptions : public ::google::protobuf::Message { public: EnumValueOptions(); virtual ~EnumValueOptions(); - + EnumValueOptions(const EnumValueOptions& from); - + inline EnumValueOptions& operator=(const EnumValueOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const EnumValueOptions& default_instance(); - + void Swap(EnumValueOptions* other); - + // implements Message ---------------------------------------------- - + EnumValueOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -1968,7 +2064,7 @@ class LIBPROTOBUF_EXPORT EnumValueOptions : public ::google::protobuf::Message { void MergeFrom(const EnumValueOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -1981,13 +2077,13 @@ class LIBPROTOBUF_EXPORT EnumValueOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -1999,24 +2095,24 @@ class LIBPROTOBUF_EXPORT EnumValueOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(EnumValueOptions) // @@protoc_insertion_point(class_scope:google.protobuf.EnumValueOptions) private: - + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static EnumValueOptions* default_instance_; }; @@ -2026,29 +2122,29 @@ class LIBPROTOBUF_EXPORT ServiceOptions : public ::google::protobuf::Message { public: ServiceOptions(); virtual ~ServiceOptions(); - + ServiceOptions(const ServiceOptions& from); - + inline ServiceOptions& operator=(const ServiceOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const ServiceOptions& default_instance(); - + void Swap(ServiceOptions* other); - + // implements Message ---------------------------------------------- - + ServiceOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2056,7 +2152,7 @@ class LIBPROTOBUF_EXPORT ServiceOptions : public ::google::protobuf::Message { void MergeFrom(const ServiceOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2069,13 +2165,13 @@ class LIBPROTOBUF_EXPORT ServiceOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -2087,24 +2183,24 @@ class LIBPROTOBUF_EXPORT ServiceOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(ServiceOptions) // @@protoc_insertion_point(class_scope:google.protobuf.ServiceOptions) private: - + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static ServiceOptions* default_instance_; }; @@ -2114,29 +2210,29 @@ class LIBPROTOBUF_EXPORT MethodOptions : public ::google::protobuf::Message { public: MethodOptions(); virtual ~MethodOptions(); - + MethodOptions(const MethodOptions& from); - + inline MethodOptions& operator=(const MethodOptions& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const MethodOptions& default_instance(); - + void Swap(MethodOptions* other); - + // implements Message ---------------------------------------------- - + MethodOptions* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2144,7 +2240,7 @@ class LIBPROTOBUF_EXPORT MethodOptions : public ::google::protobuf::Message { void MergeFrom(const MethodOptions& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2157,13 +2253,13 @@ class LIBPROTOBUF_EXPORT MethodOptions : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int uninterpreted_option_size() const; inline void clear_uninterpreted_option(); @@ -2175,24 +2271,24 @@ class LIBPROTOBUF_EXPORT MethodOptions : public ::google::protobuf::Message { uninterpreted_option() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption >* mutable_uninterpreted_option(); - + GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(MethodOptions) // @@protoc_insertion_point(class_scope:google.protobuf.MethodOptions) private: - + ::google::protobuf::internal::ExtensionSet _extensions_; - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption > uninterpreted_option_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static MethodOptions* default_instance_; }; @@ -2202,29 +2298,29 @@ class LIBPROTOBUF_EXPORT UninterpretedOption_NamePart : public ::google::protobu public: UninterpretedOption_NamePart(); virtual ~UninterpretedOption_NamePart(); - + UninterpretedOption_NamePart(const UninterpretedOption_NamePart& from); - + inline UninterpretedOption_NamePart& operator=(const UninterpretedOption_NamePart& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const UninterpretedOption_NamePart& default_instance(); - + void Swap(UninterpretedOption_NamePart* other); - + // implements Message ---------------------------------------------- - + UninterpretedOption_NamePart* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2232,7 +2328,7 @@ class LIBPROTOBUF_EXPORT UninterpretedOption_NamePart : public ::google::protobu void MergeFrom(const UninterpretedOption_NamePart& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2245,13 +2341,13 @@ class LIBPROTOBUF_EXPORT UninterpretedOption_NamePart : public ::google::protobu void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // required string name_part = 1; inline bool has_name_part() const; inline void clear_name_part(); @@ -2262,33 +2358,34 @@ class LIBPROTOBUF_EXPORT UninterpretedOption_NamePart : public ::google::protobu inline void set_name_part(const char* value, size_t size); inline ::std::string* mutable_name_part(); inline ::std::string* release_name_part(); - + inline void set_allocated_name_part(::std::string* name_part); + // required bool is_extension = 2; inline bool has_is_extension() const; inline void clear_is_extension(); static const int kIsExtensionFieldNumber = 2; inline bool is_extension() const; inline void set_is_extension(bool value); - + // @@protoc_insertion_point(class_scope:google.protobuf.UninterpretedOption.NamePart) private: inline void set_has_name_part(); inline void clear_has_name_part(); inline void set_has_is_extension(); inline void clear_has_is_extension(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::std::string* name_part_; bool is_extension_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(2 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static UninterpretedOption_NamePart* default_instance_; }; @@ -2298,29 +2395,29 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag public: UninterpretedOption(); virtual ~UninterpretedOption(); - + UninterpretedOption(const UninterpretedOption& from); - + inline UninterpretedOption& operator=(const UninterpretedOption& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const UninterpretedOption& default_instance(); - + void Swap(UninterpretedOption* other); - + // implements Message ---------------------------------------------- - + UninterpretedOption* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2328,7 +2425,7 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag void MergeFrom(const UninterpretedOption& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2341,15 +2438,15 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef UninterpretedOption_NamePart NamePart; - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.UninterpretedOption.NamePart name = 2; inline int name_size() const; inline void clear_name(); @@ -2361,7 +2458,7 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag name() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption_NamePart >* mutable_name(); - + // optional string identifier_value = 3; inline bool has_identifier_value() const; inline void clear_identifier_value(); @@ -2372,28 +2469,29 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag inline void set_identifier_value(const char* value, size_t size); inline ::std::string* mutable_identifier_value(); inline ::std::string* release_identifier_value(); - + inline void set_allocated_identifier_value(::std::string* identifier_value); + // optional uint64 positive_int_value = 4; inline bool has_positive_int_value() const; inline void clear_positive_int_value(); static const int kPositiveIntValueFieldNumber = 4; inline ::google::protobuf::uint64 positive_int_value() const; inline void set_positive_int_value(::google::protobuf::uint64 value); - + // optional int64 negative_int_value = 5; inline bool has_negative_int_value() const; inline void clear_negative_int_value(); static const int kNegativeIntValueFieldNumber = 5; inline ::google::protobuf::int64 negative_int_value() const; inline void set_negative_int_value(::google::protobuf::int64 value); - + // optional double double_value = 6; inline bool has_double_value() const; inline void clear_double_value(); static const int kDoubleValueFieldNumber = 6; inline double double_value() const; inline void set_double_value(double value); - + // optional bytes string_value = 7; inline bool has_string_value() const; inline void clear_string_value(); @@ -2404,7 +2502,8 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag inline void set_string_value(const void* value, size_t size); inline ::std::string* mutable_string_value(); inline ::std::string* release_string_value(); - + inline void set_allocated_string_value(::std::string* string_value); + // optional string aggregate_value = 8; inline bool has_aggregate_value() const; inline void clear_aggregate_value(); @@ -2415,7 +2514,8 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag inline void set_aggregate_value(const char* value, size_t size); inline ::std::string* mutable_aggregate_value(); inline ::std::string* release_aggregate_value(); - + inline void set_allocated_aggregate_value(::std::string* aggregate_value); + // @@protoc_insertion_point(class_scope:google.protobuf.UninterpretedOption) private: inline void set_has_identifier_value(); @@ -2430,9 +2530,9 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag inline void clear_has_string_value(); inline void set_has_aggregate_value(); inline void clear_has_aggregate_value(); - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::UninterpretedOption_NamePart > name_; ::std::string* identifier_value_; ::google::protobuf::uint64 positive_int_value_; @@ -2440,14 +2540,14 @@ class LIBPROTOBUF_EXPORT UninterpretedOption : public ::google::protobuf::Messag double double_value_; ::std::string* string_value_; ::std::string* aggregate_value_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(7 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static UninterpretedOption* default_instance_; }; @@ -2457,29 +2557,29 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo_Location : public ::google::protobuf::Me public: SourceCodeInfo_Location(); virtual ~SourceCodeInfo_Location(); - + SourceCodeInfo_Location(const SourceCodeInfo_Location& from); - + inline SourceCodeInfo_Location& operator=(const SourceCodeInfo_Location& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const SourceCodeInfo_Location& default_instance(); - + void Swap(SourceCodeInfo_Location* other); - + // implements Message ---------------------------------------------- - + SourceCodeInfo_Location* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2487,7 +2587,7 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo_Location : public ::google::protobuf::Me void MergeFrom(const SourceCodeInfo_Location& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2500,13 +2600,13 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo_Location : public ::google::protobuf::Me void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + // accessors ------------------------------------------------------- - + // repeated int32 path = 1 [packed = true]; inline int path_size() const; inline void clear_path(); @@ -2518,7 +2618,7 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo_Location : public ::google::protobuf::Me path() const; inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* mutable_path(); - + // repeated int32 span = 2 [packed = true]; inline int span_size() const; inline void clear_span(); @@ -2530,24 +2630,54 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo_Location : public ::google::protobuf::Me span() const; inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* mutable_span(); - + + // optional string leading_comments = 3; + inline bool has_leading_comments() const; + inline void clear_leading_comments(); + static const int kLeadingCommentsFieldNumber = 3; + inline const ::std::string& leading_comments() const; + inline void set_leading_comments(const ::std::string& value); + inline void set_leading_comments(const char* value); + inline void set_leading_comments(const char* value, size_t size); + inline ::std::string* mutable_leading_comments(); + inline ::std::string* release_leading_comments(); + inline void set_allocated_leading_comments(::std::string* leading_comments); + + // optional string trailing_comments = 4; + inline bool has_trailing_comments() const; + inline void clear_trailing_comments(); + static const int kTrailingCommentsFieldNumber = 4; + inline const ::std::string& trailing_comments() const; + inline void set_trailing_comments(const ::std::string& value); + inline void set_trailing_comments(const char* value); + inline void set_trailing_comments(const char* value, size_t size); + inline ::std::string* mutable_trailing_comments(); + inline ::std::string* release_trailing_comments(); + inline void set_allocated_trailing_comments(::std::string* trailing_comments); + // @@protoc_insertion_point(class_scope:google.protobuf.SourceCodeInfo.Location) private: - + inline void set_has_leading_comments(); + inline void clear_has_leading_comments(); + inline void set_has_trailing_comments(); + inline void clear_has_trailing_comments(); + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedField< ::google::protobuf::int32 > path_; mutable int _path_cached_byte_size_; ::google::protobuf::RepeatedField< ::google::protobuf::int32 > span_; mutable int _span_cached_byte_size_; - + ::std::string* leading_comments_; + ::std::string* trailing_comments_; + mutable int _cached_size_; - ::google::protobuf::uint32 _has_bits_[(2 + 31) / 32]; - + ::google::protobuf::uint32 _has_bits_[(4 + 31) / 32]; + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static SourceCodeInfo_Location* default_instance_; }; @@ -2557,29 +2687,29 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo : public ::google::protobuf::Message { public: SourceCodeInfo(); virtual ~SourceCodeInfo(); - + SourceCodeInfo(const SourceCodeInfo& from); - + inline SourceCodeInfo& operator=(const SourceCodeInfo& from) { CopyFrom(from); return *this; } - + inline const ::google::protobuf::UnknownFieldSet& unknown_fields() const { return _unknown_fields_; } - + inline ::google::protobuf::UnknownFieldSet* mutable_unknown_fields() { return &_unknown_fields_; } - + static const ::google::protobuf::Descriptor* descriptor(); static const SourceCodeInfo& default_instance(); - + void Swap(SourceCodeInfo* other); - + // implements Message ---------------------------------------------- - + SourceCodeInfo* New() const; void CopyFrom(const ::google::protobuf::Message& from); void MergeFrom(const ::google::protobuf::Message& from); @@ -2587,7 +2717,7 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo : public ::google::protobuf::Message { void MergeFrom(const SourceCodeInfo& from); void Clear(); bool IsInitialized() const; - + int ByteSize() const; bool MergePartialFromCodedStream( ::google::protobuf::io::CodedInputStream* input); @@ -2600,15 +2730,15 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo : public ::google::protobuf::Message { void SharedDtor(); void SetCachedSize(int size) const; public: - + ::google::protobuf::Metadata GetMetadata() const; - + // nested types ---------------------------------------------------- - + typedef SourceCodeInfo_Location Location; - + // accessors ------------------------------------------------------- - + // repeated .google.protobuf.SourceCodeInfo.Location location = 1; inline int location_size() const; inline void clear_location(); @@ -2620,21 +2750,21 @@ class LIBPROTOBUF_EXPORT SourceCodeInfo : public ::google::protobuf::Message { location() const; inline ::google::protobuf::RepeatedPtrField< ::google::protobuf::SourceCodeInfo_Location >* mutable_location(); - + // @@protoc_insertion_point(class_scope:google.protobuf.SourceCodeInfo) private: - + ::google::protobuf::UnknownFieldSet _unknown_fields_; - + ::google::protobuf::RepeatedPtrField< ::google::protobuf::SourceCodeInfo_Location > location_; - + mutable int _cached_size_; ::google::protobuf::uint32 _has_bits_[(1 + 31) / 32]; - + friend void LIBPROTOBUF_EXPORT protobuf_AddDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto(); friend void protobuf_ShutdownFile_google_2fprotobuf_2fdescriptor_2eproto(); - + void InitAsDefaultInstance(); static SourceCodeInfo* default_instance_; }; @@ -2731,6 +2861,18 @@ inline ::std::string* FileDescriptorProto::release_name() { return temp; } } +inline void FileDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string package = 2; inline bool FileDescriptorProto::has_package() const { @@ -2789,6 +2931,18 @@ inline ::std::string* FileDescriptorProto::release_package() { return temp; } } +inline void FileDescriptorProto::set_allocated_package(::std::string* package) { + if (package_ != &::google::protobuf::internal::kEmptyString) { + delete package_; + } + if (package) { + set_has_package(); + package_ = package; + } else { + clear_has_package(); + package_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated string dependency = 3; inline int FileDescriptorProto::dependency_size() const { @@ -2834,6 +2988,56 @@ FileDescriptorProto::mutable_dependency() { return &dependency_; } +// repeated int32 public_dependency = 10; +inline int FileDescriptorProto::public_dependency_size() const { + return public_dependency_.size(); +} +inline void FileDescriptorProto::clear_public_dependency() { + public_dependency_.Clear(); +} +inline ::google::protobuf::int32 FileDescriptorProto::public_dependency(int index) const { + return public_dependency_.Get(index); +} +inline void FileDescriptorProto::set_public_dependency(int index, ::google::protobuf::int32 value) { + public_dependency_.Set(index, value); +} +inline void FileDescriptorProto::add_public_dependency(::google::protobuf::int32 value) { + public_dependency_.Add(value); +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +FileDescriptorProto::public_dependency() const { + return public_dependency_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +FileDescriptorProto::mutable_public_dependency() { + return &public_dependency_; +} + +// repeated int32 weak_dependency = 11; +inline int FileDescriptorProto::weak_dependency_size() const { + return weak_dependency_.size(); +} +inline void FileDescriptorProto::clear_weak_dependency() { + weak_dependency_.Clear(); +} +inline ::google::protobuf::int32 FileDescriptorProto::weak_dependency(int index) const { + return weak_dependency_.Get(index); +} +inline void FileDescriptorProto::set_weak_dependency(int index, ::google::protobuf::int32 value) { + weak_dependency_.Set(index, value); +} +inline void FileDescriptorProto::add_weak_dependency(::google::protobuf::int32 value) { + weak_dependency_.Add(value); +} +inline const ::google::protobuf::RepeatedField< ::google::protobuf::int32 >& +FileDescriptorProto::weak_dependency() const { + return weak_dependency_; +} +inline ::google::protobuf::RepeatedField< ::google::protobuf::int32 >* +FileDescriptorProto::mutable_weak_dependency() { + return &weak_dependency_; +} + // repeated .google.protobuf.DescriptorProto message_type = 4; inline int FileDescriptorProto::message_type_size() const { return message_type_.size(); @@ -2936,13 +3140,13 @@ FileDescriptorProto::mutable_extension() { // optional .google.protobuf.FileOptions options = 8; inline bool FileDescriptorProto::has_options() const { - return (_has_bits_[0] & 0x00000080u) != 0; + return (_has_bits_[0] & 0x00000200u) != 0; } inline void FileDescriptorProto::set_has_options() { - _has_bits_[0] |= 0x00000080u; + _has_bits_[0] |= 0x00000200u; } inline void FileDescriptorProto::clear_has_options() { - _has_bits_[0] &= ~0x00000080u; + _has_bits_[0] &= ~0x00000200u; } inline void FileDescriptorProto::clear_options() { if (options_ != NULL) options_->::google::protobuf::FileOptions::Clear(); @@ -2962,16 +3166,25 @@ inline ::google::protobuf::FileOptions* FileDescriptorProto::release_options() { options_ = NULL; return temp; } +inline void FileDescriptorProto::set_allocated_options(::google::protobuf::FileOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // optional .google.protobuf.SourceCodeInfo source_code_info = 9; inline bool FileDescriptorProto::has_source_code_info() const { - return (_has_bits_[0] & 0x00000100u) != 0; + return (_has_bits_[0] & 0x00000400u) != 0; } inline void FileDescriptorProto::set_has_source_code_info() { - _has_bits_[0] |= 0x00000100u; + _has_bits_[0] |= 0x00000400u; } inline void FileDescriptorProto::clear_has_source_code_info() { - _has_bits_[0] &= ~0x00000100u; + _has_bits_[0] &= ~0x00000400u; } inline void FileDescriptorProto::clear_source_code_info() { if (source_code_info_ != NULL) source_code_info_->::google::protobuf::SourceCodeInfo::Clear(); @@ -2991,6 +3204,15 @@ inline ::google::protobuf::SourceCodeInfo* FileDescriptorProto::release_source_c source_code_info_ = NULL; return temp; } +inline void FileDescriptorProto::set_allocated_source_code_info(::google::protobuf::SourceCodeInfo* source_code_info) { + delete source_code_info_; + source_code_info_ = source_code_info; + if (source_code_info) { + set_has_source_code_info(); + } else { + clear_has_source_code_info(); + } +} // ------------------------------------------------------------------- @@ -3101,6 +3323,18 @@ inline ::std::string* DescriptorProto::release_name() { return temp; } } +inline void DescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated .google.protobuf.FieldDescriptorProto field = 2; inline int DescriptorProto::field_size() const { @@ -3255,6 +3489,15 @@ inline ::google::protobuf::MessageOptions* DescriptorProto::release_options() { options_ = NULL; return temp; } +inline void DescriptorProto::set_allocated_options(::google::protobuf::MessageOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -3317,6 +3560,18 @@ inline ::std::string* FieldDescriptorProto::release_name() { return temp; } } +inline void FieldDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional int32 number = 3; inline bool FieldDescriptorProto::has_number() const { @@ -3358,7 +3613,7 @@ inline ::google::protobuf::FieldDescriptorProto_Label FieldDescriptorProto::labe return static_cast< ::google::protobuf::FieldDescriptorProto_Label >(label_); } inline void FieldDescriptorProto::set_label(::google::protobuf::FieldDescriptorProto_Label value) { - GOOGLE_DCHECK(::google::protobuf::FieldDescriptorProto_Label_IsValid(value)); + assert(::google::protobuf::FieldDescriptorProto_Label_IsValid(value)); set_has_label(); label_ = value; } @@ -3381,7 +3636,7 @@ inline ::google::protobuf::FieldDescriptorProto_Type FieldDescriptorProto::type( return static_cast< ::google::protobuf::FieldDescriptorProto_Type >(type_); } inline void FieldDescriptorProto::set_type(::google::protobuf::FieldDescriptorProto_Type value) { - GOOGLE_DCHECK(::google::protobuf::FieldDescriptorProto_Type_IsValid(value)); + assert(::google::protobuf::FieldDescriptorProto_Type_IsValid(value)); set_has_type(); type_ = value; } @@ -3443,6 +3698,18 @@ inline ::std::string* FieldDescriptorProto::release_type_name() { return temp; } } +inline void FieldDescriptorProto::set_allocated_type_name(::std::string* type_name) { + if (type_name_ != &::google::protobuf::internal::kEmptyString) { + delete type_name_; + } + if (type_name) { + set_has_type_name(); + type_name_ = type_name; + } else { + clear_has_type_name(); + type_name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string extendee = 2; inline bool FieldDescriptorProto::has_extendee() const { @@ -3501,6 +3768,18 @@ inline ::std::string* FieldDescriptorProto::release_extendee() { return temp; } } +inline void FieldDescriptorProto::set_allocated_extendee(::std::string* extendee) { + if (extendee_ != &::google::protobuf::internal::kEmptyString) { + delete extendee_; + } + if (extendee) { + set_has_extendee(); + extendee_ = extendee; + } else { + clear_has_extendee(); + extendee_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string default_value = 7; inline bool FieldDescriptorProto::has_default_value() const { @@ -3559,6 +3838,18 @@ inline ::std::string* FieldDescriptorProto::release_default_value() { return temp; } } +inline void FieldDescriptorProto::set_allocated_default_value(::std::string* default_value) { + if (default_value_ != &::google::protobuf::internal::kEmptyString) { + delete default_value_; + } + if (default_value) { + set_has_default_value(); + default_value_ = default_value; + } else { + clear_has_default_value(); + default_value_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional .google.protobuf.FieldOptions options = 8; inline bool FieldDescriptorProto::has_options() const { @@ -3588,6 +3879,15 @@ inline ::google::protobuf::FieldOptions* FieldDescriptorProto::release_options() options_ = NULL; return temp; } +inline void FieldDescriptorProto::set_allocated_options(::google::protobuf::FieldOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -3650,6 +3950,18 @@ inline ::std::string* EnumDescriptorProto::release_name() { return temp; } } +inline void EnumDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated .google.protobuf.EnumValueDescriptorProto value = 2; inline int EnumDescriptorProto::value_size() const { @@ -3704,6 +4016,15 @@ inline ::google::protobuf::EnumOptions* EnumDescriptorProto::release_options() { options_ = NULL; return temp; } +inline void EnumDescriptorProto::set_allocated_options(::google::protobuf::EnumOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -3766,6 +4087,18 @@ inline ::std::string* EnumValueDescriptorProto::release_name() { return temp; } } +inline void EnumValueDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional int32 number = 2; inline bool EnumValueDescriptorProto::has_number() const { @@ -3817,6 +4150,15 @@ inline ::google::protobuf::EnumValueOptions* EnumValueDescriptorProto::release_o options_ = NULL; return temp; } +inline void EnumValueDescriptorProto::set_allocated_options(::google::protobuf::EnumValueOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -3879,6 +4221,18 @@ inline ::std::string* ServiceDescriptorProto::release_name() { return temp; } } +inline void ServiceDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // repeated .google.protobuf.MethodDescriptorProto method = 2; inline int ServiceDescriptorProto::method_size() const { @@ -3933,6 +4287,15 @@ inline ::google::protobuf::ServiceOptions* ServiceDescriptorProto::release_optio options_ = NULL; return temp; } +inline void ServiceDescriptorProto::set_allocated_options(::google::protobuf::ServiceOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -3995,6 +4358,18 @@ inline ::std::string* MethodDescriptorProto::release_name() { return temp; } } +inline void MethodDescriptorProto::set_allocated_name(::std::string* name) { + if (name_ != &::google::protobuf::internal::kEmptyString) { + delete name_; + } + if (name) { + set_has_name(); + name_ = name; + } else { + clear_has_name(); + name_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string input_type = 2; inline bool MethodDescriptorProto::has_input_type() const { @@ -4053,6 +4428,18 @@ inline ::std::string* MethodDescriptorProto::release_input_type() { return temp; } } +inline void MethodDescriptorProto::set_allocated_input_type(::std::string* input_type) { + if (input_type_ != &::google::protobuf::internal::kEmptyString) { + delete input_type_; + } + if (input_type) { + set_has_input_type(); + input_type_ = input_type; + } else { + clear_has_input_type(); + input_type_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string output_type = 3; inline bool MethodDescriptorProto::has_output_type() const { @@ -4111,6 +4498,18 @@ inline ::std::string* MethodDescriptorProto::release_output_type() { return temp; } } +inline void MethodDescriptorProto::set_allocated_output_type(::std::string* output_type) { + if (output_type_ != &::google::protobuf::internal::kEmptyString) { + delete output_type_; + } + if (output_type) { + set_has_output_type(); + output_type_ = output_type; + } else { + clear_has_output_type(); + output_type_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional .google.protobuf.MethodOptions options = 4; inline bool MethodDescriptorProto::has_options() const { @@ -4140,6 +4539,15 @@ inline ::google::protobuf::MethodOptions* MethodDescriptorProto::release_options options_ = NULL; return temp; } +inline void MethodDescriptorProto::set_allocated_options(::google::protobuf::MethodOptions* options) { + delete options_; + options_ = options; + if (options) { + set_has_options(); + } else { + clear_has_options(); + } +} // ------------------------------------------------------------------- @@ -4202,6 +4610,18 @@ inline ::std::string* FileOptions::release_java_package() { return temp; } } +inline void FileOptions::set_allocated_java_package(::std::string* java_package) { + if (java_package_ != &::google::protobuf::internal::kEmptyString) { + delete java_package_; + } + if (java_package) { + set_has_java_package(); + java_package_ = java_package; + } else { + clear_has_java_package(); + java_package_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string java_outer_classname = 8; inline bool FileOptions::has_java_outer_classname() const { @@ -4260,6 +4680,18 @@ inline ::std::string* FileOptions::release_java_outer_classname() { return temp; } } +inline void FileOptions::set_allocated_java_outer_classname(::std::string* java_outer_classname) { + if (java_outer_classname_ != &::google::protobuf::internal::kEmptyString) { + delete java_outer_classname_; + } + if (java_outer_classname) { + set_has_java_outer_classname(); + java_outer_classname_ = java_outer_classname; + } else { + clear_has_java_outer_classname(); + java_outer_classname_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional bool java_multiple_files = 10 [default = false]; inline bool FileOptions::has_java_multiple_files() const { @@ -4283,7 +4715,7 @@ inline void FileOptions::set_java_multiple_files(bool value) { java_multiple_files_ = value; } -// optional bool retain_unknown_fields = 11 [default = false]; +// optional bool retain_unknown_fields = 12 [default = false]; inline bool FileOptions::has_retain_unknown_fields() const { return (_has_bits_[0] & 0x00000008u) != 0; } @@ -4345,20 +4777,90 @@ inline ::google::protobuf::FileOptions_OptimizeMode FileOptions::optimize_for() return static_cast< ::google::protobuf::FileOptions_OptimizeMode >(optimize_for_); } inline void FileOptions::set_optimize_for(::google::protobuf::FileOptions_OptimizeMode value) { - GOOGLE_DCHECK(::google::protobuf::FileOptions_OptimizeMode_IsValid(value)); + assert(::google::protobuf::FileOptions_OptimizeMode_IsValid(value)); set_has_optimize_for(); optimize_for_ = value; } +// optional string go_package = 11; +inline bool FileOptions::has_go_package() const { + return (_has_bits_[0] & 0x00000040u) != 0; +} +inline void FileOptions::set_has_go_package() { + _has_bits_[0] |= 0x00000040u; +} +inline void FileOptions::clear_has_go_package() { + _has_bits_[0] &= ~0x00000040u; +} +inline void FileOptions::clear_go_package() { + if (go_package_ != &::google::protobuf::internal::kEmptyString) { + go_package_->clear(); + } + clear_has_go_package(); +} +inline const ::std::string& FileOptions::go_package() const { + return *go_package_; +} +inline void FileOptions::set_go_package(const ::std::string& value) { + set_has_go_package(); + if (go_package_ == &::google::protobuf::internal::kEmptyString) { + go_package_ = new ::std::string; + } + go_package_->assign(value); +} +inline void FileOptions::set_go_package(const char* value) { + set_has_go_package(); + if (go_package_ == &::google::protobuf::internal::kEmptyString) { + go_package_ = new ::std::string; + } + go_package_->assign(value); +} +inline void FileOptions::set_go_package(const char* value, size_t size) { + set_has_go_package(); + if (go_package_ == &::google::protobuf::internal::kEmptyString) { + go_package_ = new ::std::string; + } + go_package_->assign(reinterpret_cast<const char*>(value), size); +} +inline ::std::string* FileOptions::mutable_go_package() { + set_has_go_package(); + if (go_package_ == &::google::protobuf::internal::kEmptyString) { + go_package_ = new ::std::string; + } + return go_package_; +} +inline ::std::string* FileOptions::release_go_package() { + clear_has_go_package(); + if (go_package_ == &::google::protobuf::internal::kEmptyString) { + return NULL; + } else { + ::std::string* temp = go_package_; + go_package_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + return temp; + } +} +inline void FileOptions::set_allocated_go_package(::std::string* go_package) { + if (go_package_ != &::google::protobuf::internal::kEmptyString) { + delete go_package_; + } + if (go_package) { + set_has_go_package(); + go_package_ = go_package; + } else { + clear_has_go_package(); + go_package_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} + // optional bool cc_generic_services = 16 [default = false]; inline bool FileOptions::has_cc_generic_services() const { - return (_has_bits_[0] & 0x00000040u) != 0; + return (_has_bits_[0] & 0x00000080u) != 0; } inline void FileOptions::set_has_cc_generic_services() { - _has_bits_[0] |= 0x00000040u; + _has_bits_[0] |= 0x00000080u; } inline void FileOptions::clear_has_cc_generic_services() { - _has_bits_[0] &= ~0x00000040u; + _has_bits_[0] &= ~0x00000080u; } inline void FileOptions::clear_cc_generic_services() { cc_generic_services_ = false; @@ -4374,13 +4876,13 @@ inline void FileOptions::set_cc_generic_services(bool value) { // optional bool java_generic_services = 17 [default = false]; inline bool FileOptions::has_java_generic_services() const { - return (_has_bits_[0] & 0x00000080u) != 0; + return (_has_bits_[0] & 0x00000100u) != 0; } inline void FileOptions::set_has_java_generic_services() { - _has_bits_[0] |= 0x00000080u; + _has_bits_[0] |= 0x00000100u; } inline void FileOptions::clear_has_java_generic_services() { - _has_bits_[0] &= ~0x00000080u; + _has_bits_[0] &= ~0x00000100u; } inline void FileOptions::clear_java_generic_services() { java_generic_services_ = false; @@ -4396,13 +4898,13 @@ inline void FileOptions::set_java_generic_services(bool value) { // optional bool py_generic_services = 18 [default = false]; inline bool FileOptions::has_py_generic_services() const { - return (_has_bits_[0] & 0x00000100u) != 0; + return (_has_bits_[0] & 0x00000200u) != 0; } inline void FileOptions::set_has_py_generic_services() { - _has_bits_[0] |= 0x00000100u; + _has_bits_[0] |= 0x00000200u; } inline void FileOptions::clear_has_py_generic_services() { - _has_bits_[0] &= ~0x00000100u; + _has_bits_[0] &= ~0x00000200u; } inline void FileOptions::clear_py_generic_services() { py_generic_services_ = false; @@ -4536,7 +5038,7 @@ inline ::google::protobuf::FieldOptions_CType FieldOptions::ctype() const { return static_cast< ::google::protobuf::FieldOptions_CType >(ctype_); } inline void FieldOptions::set_ctype(::google::protobuf::FieldOptions_CType value) { - GOOGLE_DCHECK(::google::protobuf::FieldOptions_CType_IsValid(value)); + assert(::google::protobuf::FieldOptions_CType_IsValid(value)); set_has_ctype(); ctype_ = value; } @@ -4563,15 +5065,37 @@ inline void FieldOptions::set_packed(bool value) { packed_ = value; } +// optional bool lazy = 5 [default = false]; +inline bool FieldOptions::has_lazy() const { + return (_has_bits_[0] & 0x00000004u) != 0; +} +inline void FieldOptions::set_has_lazy() { + _has_bits_[0] |= 0x00000004u; +} +inline void FieldOptions::clear_has_lazy() { + _has_bits_[0] &= ~0x00000004u; +} +inline void FieldOptions::clear_lazy() { + lazy_ = false; + clear_has_lazy(); +} +inline bool FieldOptions::lazy() const { + return lazy_; +} +inline void FieldOptions::set_lazy(bool value) { + set_has_lazy(); + lazy_ = value; +} + // optional bool deprecated = 3 [default = false]; inline bool FieldOptions::has_deprecated() const { - return (_has_bits_[0] & 0x00000004u) != 0; + return (_has_bits_[0] & 0x00000008u) != 0; } inline void FieldOptions::set_has_deprecated() { - _has_bits_[0] |= 0x00000004u; + _has_bits_[0] |= 0x00000008u; } inline void FieldOptions::clear_has_deprecated() { - _has_bits_[0] &= ~0x00000004u; + _has_bits_[0] &= ~0x00000008u; } inline void FieldOptions::clear_deprecated() { deprecated_ = false; @@ -4587,13 +5111,13 @@ inline void FieldOptions::set_deprecated(bool value) { // optional string experimental_map_key = 9; inline bool FieldOptions::has_experimental_map_key() const { - return (_has_bits_[0] & 0x00000008u) != 0; + return (_has_bits_[0] & 0x00000010u) != 0; } inline void FieldOptions::set_has_experimental_map_key() { - _has_bits_[0] |= 0x00000008u; + _has_bits_[0] |= 0x00000010u; } inline void FieldOptions::clear_has_experimental_map_key() { - _has_bits_[0] &= ~0x00000008u; + _has_bits_[0] &= ~0x00000010u; } inline void FieldOptions::clear_experimental_map_key() { if (experimental_map_key_ != &::google::protobuf::internal::kEmptyString) { @@ -4642,6 +5166,40 @@ inline ::std::string* FieldOptions::release_experimental_map_key() { return temp; } } +inline void FieldOptions::set_allocated_experimental_map_key(::std::string* experimental_map_key) { + if (experimental_map_key_ != &::google::protobuf::internal::kEmptyString) { + delete experimental_map_key_; + } + if (experimental_map_key) { + set_has_experimental_map_key(); + experimental_map_key_ = experimental_map_key; + } else { + clear_has_experimental_map_key(); + experimental_map_key_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} + +// optional bool weak = 10 [default = false]; +inline bool FieldOptions::has_weak() const { + return (_has_bits_[0] & 0x00000020u) != 0; +} +inline void FieldOptions::set_has_weak() { + _has_bits_[0] |= 0x00000020u; +} +inline void FieldOptions::clear_has_weak() { + _has_bits_[0] &= ~0x00000020u; +} +inline void FieldOptions::clear_weak() { + weak_ = false; + clear_has_weak(); +} +inline bool FieldOptions::weak() const { + return weak_; +} +inline void FieldOptions::set_weak(bool value) { + set_has_weak(); + weak_ = value; +} // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int FieldOptions::uninterpreted_option_size() const { @@ -4672,6 +5230,28 @@ FieldOptions::mutable_uninterpreted_option() { // EnumOptions +// optional bool allow_alias = 2 [default = true]; +inline bool EnumOptions::has_allow_alias() const { + return (_has_bits_[0] & 0x00000001u) != 0; +} +inline void EnumOptions::set_has_allow_alias() { + _has_bits_[0] |= 0x00000001u; +} +inline void EnumOptions::clear_has_allow_alias() { + _has_bits_[0] &= ~0x00000001u; +} +inline void EnumOptions::clear_allow_alias() { + allow_alias_ = true; + clear_has_allow_alias(); +} +inline bool EnumOptions::allow_alias() const { + return allow_alias_; +} +inline void EnumOptions::set_allow_alias(bool value) { + set_has_allow_alias(); + allow_alias_ = value; +} + // repeated .google.protobuf.UninterpretedOption uninterpreted_option = 999; inline int EnumOptions::uninterpreted_option_size() const { return uninterpreted_option_.size(); @@ -4845,6 +5425,18 @@ inline ::std::string* UninterpretedOption_NamePart::release_name_part() { return temp; } } +inline void UninterpretedOption_NamePart::set_allocated_name_part(::std::string* name_part) { + if (name_part_ != &::google::protobuf::internal::kEmptyString) { + delete name_part_; + } + if (name_part) { + set_has_name_part(); + name_part_ = name_part; + } else { + clear_has_name_part(); + name_part_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // required bool is_extension = 2; inline bool UninterpretedOption_NamePart::has_is_extension() const { @@ -4954,6 +5546,18 @@ inline ::std::string* UninterpretedOption::release_identifier_value() { return temp; } } +inline void UninterpretedOption::set_allocated_identifier_value(::std::string* identifier_value) { + if (identifier_value_ != &::google::protobuf::internal::kEmptyString) { + delete identifier_value_; + } + if (identifier_value) { + set_has_identifier_value(); + identifier_value_ = identifier_value; + } else { + clear_has_identifier_value(); + identifier_value_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional uint64 positive_int_value = 4; inline bool UninterpretedOption::has_positive_int_value() const { @@ -5078,6 +5682,18 @@ inline ::std::string* UninterpretedOption::release_string_value() { return temp; } } +inline void UninterpretedOption::set_allocated_string_value(::std::string* string_value) { + if (string_value_ != &::google::protobuf::internal::kEmptyString) { + delete string_value_; + } + if (string_value) { + set_has_string_value(); + string_value_ = string_value; + } else { + clear_has_string_value(); + string_value_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // optional string aggregate_value = 8; inline bool UninterpretedOption::has_aggregate_value() const { @@ -5136,6 +5752,18 @@ inline ::std::string* UninterpretedOption::release_aggregate_value() { return temp; } } +inline void UninterpretedOption::set_allocated_aggregate_value(::std::string* aggregate_value) { + if (aggregate_value_ != &::google::protobuf::internal::kEmptyString) { + delete aggregate_value_; + } + if (aggregate_value) { + set_has_aggregate_value(); + aggregate_value_ = aggregate_value; + } else { + clear_has_aggregate_value(); + aggregate_value_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} // ------------------------------------------------------------------- @@ -5191,6 +5819,146 @@ SourceCodeInfo_Location::mutable_span() { return &span_; } +// optional string leading_comments = 3; +inline bool SourceCodeInfo_Location::has_leading_comments() const { + return (_has_bits_[0] & 0x00000004u) != 0; +} +inline void SourceCodeInfo_Location::set_has_leading_comments() { + _has_bits_[0] |= 0x00000004u; +} +inline void SourceCodeInfo_Location::clear_has_leading_comments() { + _has_bits_[0] &= ~0x00000004u; +} +inline void SourceCodeInfo_Location::clear_leading_comments() { + if (leading_comments_ != &::google::protobuf::internal::kEmptyString) { + leading_comments_->clear(); + } + clear_has_leading_comments(); +} +inline const ::std::string& SourceCodeInfo_Location::leading_comments() const { + return *leading_comments_; +} +inline void SourceCodeInfo_Location::set_leading_comments(const ::std::string& value) { + set_has_leading_comments(); + if (leading_comments_ == &::google::protobuf::internal::kEmptyString) { + leading_comments_ = new ::std::string; + } + leading_comments_->assign(value); +} +inline void SourceCodeInfo_Location::set_leading_comments(const char* value) { + set_has_leading_comments(); + if (leading_comments_ == &::google::protobuf::internal::kEmptyString) { + leading_comments_ = new ::std::string; + } + leading_comments_->assign(value); +} +inline void SourceCodeInfo_Location::set_leading_comments(const char* value, size_t size) { + set_has_leading_comments(); + if (leading_comments_ == &::google::protobuf::internal::kEmptyString) { + leading_comments_ = new ::std::string; + } + leading_comments_->assign(reinterpret_cast<const char*>(value), size); +} +inline ::std::string* SourceCodeInfo_Location::mutable_leading_comments() { + set_has_leading_comments(); + if (leading_comments_ == &::google::protobuf::internal::kEmptyString) { + leading_comments_ = new ::std::string; + } + return leading_comments_; +} +inline ::std::string* SourceCodeInfo_Location::release_leading_comments() { + clear_has_leading_comments(); + if (leading_comments_ == &::google::protobuf::internal::kEmptyString) { + return NULL; + } else { + ::std::string* temp = leading_comments_; + leading_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + return temp; + } +} +inline void SourceCodeInfo_Location::set_allocated_leading_comments(::std::string* leading_comments) { + if (leading_comments_ != &::google::protobuf::internal::kEmptyString) { + delete leading_comments_; + } + if (leading_comments) { + set_has_leading_comments(); + leading_comments_ = leading_comments; + } else { + clear_has_leading_comments(); + leading_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} + +// optional string trailing_comments = 4; +inline bool SourceCodeInfo_Location::has_trailing_comments() const { + return (_has_bits_[0] & 0x00000008u) != 0; +} +inline void SourceCodeInfo_Location::set_has_trailing_comments() { + _has_bits_[0] |= 0x00000008u; +} +inline void SourceCodeInfo_Location::clear_has_trailing_comments() { + _has_bits_[0] &= ~0x00000008u; +} +inline void SourceCodeInfo_Location::clear_trailing_comments() { + if (trailing_comments_ != &::google::protobuf::internal::kEmptyString) { + trailing_comments_->clear(); + } + clear_has_trailing_comments(); +} +inline const ::std::string& SourceCodeInfo_Location::trailing_comments() const { + return *trailing_comments_; +} +inline void SourceCodeInfo_Location::set_trailing_comments(const ::std::string& value) { + set_has_trailing_comments(); + if (trailing_comments_ == &::google::protobuf::internal::kEmptyString) { + trailing_comments_ = new ::std::string; + } + trailing_comments_->assign(value); +} +inline void SourceCodeInfo_Location::set_trailing_comments(const char* value) { + set_has_trailing_comments(); + if (trailing_comments_ == &::google::protobuf::internal::kEmptyString) { + trailing_comments_ = new ::std::string; + } + trailing_comments_->assign(value); +} +inline void SourceCodeInfo_Location::set_trailing_comments(const char* value, size_t size) { + set_has_trailing_comments(); + if (trailing_comments_ == &::google::protobuf::internal::kEmptyString) { + trailing_comments_ = new ::std::string; + } + trailing_comments_->assign(reinterpret_cast<const char*>(value), size); +} +inline ::std::string* SourceCodeInfo_Location::mutable_trailing_comments() { + set_has_trailing_comments(); + if (trailing_comments_ == &::google::protobuf::internal::kEmptyString) { + trailing_comments_ = new ::std::string; + } + return trailing_comments_; +} +inline ::std::string* SourceCodeInfo_Location::release_trailing_comments() { + clear_has_trailing_comments(); + if (trailing_comments_ == &::google::protobuf::internal::kEmptyString) { + return NULL; + } else { + ::std::string* temp = trailing_comments_; + trailing_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + return temp; + } +} +inline void SourceCodeInfo_Location::set_allocated_trailing_comments(::std::string* trailing_comments) { + if (trailing_comments_ != &::google::protobuf::internal::kEmptyString) { + delete trailing_comments_; + } + if (trailing_comments) { + set_has_trailing_comments(); + trailing_comments_ = trailing_comments; + } else { + clear_has_trailing_comments(); + trailing_comments_ = const_cast< ::std::string*>(&::google::protobuf::internal::kEmptyString); + } +} + // ------------------------------------------------------------------- // SourceCodeInfo diff --git a/third_party/protobuf/src/google/protobuf/descriptor.proto b/third_party/protobuf/src/google/protobuf/descriptor.proto index bd3cb96..ef9eb65 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor.proto +++ b/third_party/protobuf/src/google/protobuf/descriptor.proto @@ -59,6 +59,11 @@ message FileDescriptorProto { // Names of files imported by this file. repeated string dependency = 3; + // Indexes of the public imported files in the dependency list above. + repeated int32 public_dependency = 10; + // Indexes of the weak imported files in the dependency list. + // For Google-internal migration only. Do not use. + repeated int32 weak_dependency = 11; // All top-level definitions in this file. repeated DescriptorProto message_type = 4; @@ -101,13 +106,13 @@ message FieldDescriptorProto { // Order is weird for historical reasons. TYPE_DOUBLE = 1; TYPE_FLOAT = 2; - TYPE_INT64 = 3; // Not ZigZag encoded. Negative numbers - // take 10 bytes. Use TYPE_SINT64 if negative - // values are likely. + // Not ZigZag encoded. Negative numbers take 10 bytes. Use TYPE_SINT64 if + // negative values are likely. + TYPE_INT64 = 3; TYPE_UINT64 = 4; - TYPE_INT32 = 5; // Not ZigZag encoded. Negative numbers - // take 10 bytes. Use TYPE_SINT32 if negative - // values are likely. + // Not ZigZag encoded. Negative numbers take 10 bytes. Use TYPE_SINT32 if + // negative values are likely. + TYPE_INT32 = 5; TYPE_FIXED64 = 6; TYPE_FIXED32 = 7; TYPE_BOOL = 8; @@ -199,6 +204,7 @@ message MethodDescriptorProto { optional MethodOptions options = 4; } + // =================================================================== // Options @@ -260,7 +266,7 @@ message FileOptions { // reserializing a message will retain them. This is the default behaviour // unless LITE_RUNTIME is specified. Therefore, this option only makes sense // when LITE_RUNTIME is in use. - optional bool retain_unknown_fields = 11 [default=false]; + optional bool retain_unknown_fields = 12 [default=false]; // If set true, then the Java code generator will generate equals() and // hashCode() methods for all messages defined in the .proto file. This is @@ -277,6 +283,9 @@ message FileOptions { } optional OptimizeMode optimize_for = 9 [default=SPEED]; + // Sets the Go package where structs generated from this .proto will be + // placed. There is no default. + optional string go_package = 11; @@ -355,6 +364,37 @@ message FieldOptions { optional bool packed = 2; + + // Should this field be parsed lazily? Lazy applies only to message-type + // fields. It means that when the outer message is initially parsed, the + // inner message's contents will not be parsed but instead stored in encoded + // form. The inner message will actually be parsed when it is first accessed. + // + // This is only a hint. Implementations are free to choose whether to use + // eager or lazy parsing regardless of the value of this option. However, + // setting this option true suggests that the protocol author believes that + // using lazy parsing on this field is worth the additional bookkeeping + // overhead typically needed to implement it. + // + // This option does not affect the public interface of any generated code; + // all method signatures remain the same. Furthermore, thread-safety of the + // interface is not affected by this option; const methods remain safe to + // call from multiple threads concurrently, while non-const methods continue + // to require exclusive access. + // + // + // Note that implementations may choose not to check required fields within + // a lazy sub-message. That is, calling IsInitialized() on the outher message + // may return true even if the inner message has missing required fields. + // This is necessary because otherwise the inner message would have to be + // parsed in order to perform the check, defeating the purpose of lazy + // parsing. An implementation which chooses not to check required fields + // must be consistent about it. That is, for any particular sub-message, the + // implementation must either *always* check its required fields, or *never* + // check its required fields, regardless of whether or not the message has + // been parsed. + optional bool lazy = 5 [default=false]; + // Is this field deprecated? // Depending on the target platform, this can emit Deprecated annotations // for accessors, or it will be completely ignored; in the very least, this @@ -375,6 +415,9 @@ message FieldOptions { // TODO: Fully-implement this, then remove the "experimental_" prefix. optional string experimental_map_key = 9; + // For Google-internal migration only. Do not use. + optional bool weak = 10 [default=false]; + // The parser stores options it doesn't recognize here. See above. repeated UninterpretedOption uninterpreted_option = 999; @@ -384,6 +427,10 @@ message FieldOptions { message EnumOptions { + // Set this option to false to disallow mapping different tag names to a same + // value. + optional bool allow_alias = 2 [default=true]; + // The parser stores options it doesn't recognize here. See above. repeated UninterpretedOption uninterpreted_option = 999; @@ -427,6 +474,7 @@ message MethodOptions { extensions 1000 to max; } + // A message representing a option the parser does not recognize. This only // appears in options protos created by the compiler::Parser class. // DescriptorPool resolves these when building Descriptor objects. Therefore, @@ -538,7 +586,41 @@ message SourceCodeInfo { // 1 to each before displaying to a user. repeated int32 span = 2 [packed=true]; - // TODO(kenton): Record comments appearing before and after the - // declaration. + // If this SourceCodeInfo represents a complete declaration, these are any + // comments appearing before and after the declaration which appear to be + // attached to the declaration. + // + // A series of line comments appearing on consecutive lines, with no other + // tokens appearing on those lines, will be treated as a single comment. + // + // Only the comment content is provided; comment markers (e.g. //) are + // stripped out. For block comments, leading whitespace and an asterisk + // will be stripped from the beginning of each line other than the first. + // Newlines are included in the output. + // + // Examples: + // + // optional int32 foo = 1; // Comment attached to foo. + // // Comment attached to bar. + // optional int32 bar = 2; + // + // optional string baz = 3; + // // Comment attached to baz. + // // Another line attached to baz. + // + // // Comment attached to qux. + // // + // // Another line attached to qux. + // optional double qux = 4; + // + // optional string corge = 5; + // /* Block comment attached + // * to corge. Leading asterisks + // * will be removed. */ + // /* Block comment attached to + // * grault. */ + // optional int32 grault = 6; + optional string leading_comments = 3; + optional string trailing_comments = 4; } } diff --git a/third_party/protobuf/src/google/protobuf/descriptor_database.cc b/third_party/protobuf/src/google/protobuf/descriptor_database.cc index 23e48a6..35e459d 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor_database.cc +++ b/third_party/protobuf/src/google/protobuf/descriptor_database.cc @@ -39,7 +39,7 @@ #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/wire_format_lite_inl.h> #include <google/protobuf/stubs/strutil.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> #include <google/protobuf/stubs/map-util.h> namespace google { diff --git a/third_party/protobuf/src/google/protobuf/descriptor_database.h b/third_party/protobuf/src/google/protobuf/descriptor_database.h index f32b1db..2ccb145 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor_database.h +++ b/third_party/protobuf/src/google/protobuf/descriptor_database.h @@ -41,6 +41,7 @@ #include <string> #include <utility> #include <vector> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/descriptor.h> namespace google { diff --git a/third_party/protobuf/src/google/protobuf/descriptor_unittest.cc b/third_party/protobuf/src/google/protobuf/descriptor_unittest.cc index 55aebfd..3503f13 100644 --- a/third_party/protobuf/src/google/protobuf/descriptor_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/descriptor_unittest.cc @@ -36,13 +36,15 @@ #include <vector> +#include <google/protobuf/compiler/importer.h> +#include <google/protobuf/unittest.pb.h> +#include <google/protobuf/unittest_custom_options.pb.h> +#include <google/protobuf/io/zero_copy_stream_impl.h> +#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/descriptor.h> #include <google/protobuf/descriptor_database.h> #include <google/protobuf/dynamic_message.h> -#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/text_format.h> -#include <google/protobuf/unittest.pb.h> -#include <google/protobuf/unittest_custom_options.pb.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/substitute.h> @@ -1518,9 +1520,8 @@ TEST_F(ExtensionDescriptorTest, FindAllExtensions) { class MiscTest : public testing::Test { protected: - // Function which makes a field of the given type just to find out what its - // cpp_type is. - FieldDescriptor::CppType GetCppTypeForFieldType(FieldDescriptor::Type type) { + // Function which makes a field descriptor of the given type. + const FieldDescriptor* GetFieldDescriptorOfType(FieldDescriptor::Type type) { FileDescriptorProto file_proto; file_proto.set_name("foo.proto"); AddEmptyEnum(&file_proto, "DummyEnum"); @@ -1538,19 +1539,62 @@ class MiscTest : public testing::Test { } // Build the descriptors and get the pointers. - DescriptorPool pool; - const FileDescriptor* file = pool.BuildFile(file_proto); + pool_.reset(new DescriptorPool()); + const FileDescriptor* file = pool_->BuildFile(file_proto); if (file != NULL && file->message_type_count() == 1 && file->message_type(0)->field_count() == 1) { - return file->message_type(0)->field(0)->cpp_type(); + return file->message_type(0)->field(0); } else { - return static_cast<FieldDescriptor::CppType>(0); + return NULL; } } + + const char* GetTypeNameForFieldType(FieldDescriptor::Type type) { + const FieldDescriptor* field = GetFieldDescriptorOfType(type); + return field != NULL ? field->type_name() : ""; + } + + FieldDescriptor::CppType GetCppTypeForFieldType(FieldDescriptor::Type type) { + const FieldDescriptor* field = GetFieldDescriptorOfType(type); + return field != NULL ? field->cpp_type() : + static_cast<FieldDescriptor::CppType>(0); + } + + const char* GetCppTypeNameForFieldType(FieldDescriptor::Type type) { + const FieldDescriptor* field = GetFieldDescriptorOfType(type); + return field != NULL ? field->cpp_type_name() : ""; + } + + scoped_ptr<DescriptorPool> pool_; }; +TEST_F(MiscTest, TypeNames) { + // Test that correct type names are returned. + + typedef FieldDescriptor FD; // avoid ugly line wrapping + + EXPECT_STREQ("double" , GetTypeNameForFieldType(FD::TYPE_DOUBLE )); + EXPECT_STREQ("float" , GetTypeNameForFieldType(FD::TYPE_FLOAT )); + EXPECT_STREQ("int64" , GetTypeNameForFieldType(FD::TYPE_INT64 )); + EXPECT_STREQ("uint64" , GetTypeNameForFieldType(FD::TYPE_UINT64 )); + EXPECT_STREQ("int32" , GetTypeNameForFieldType(FD::TYPE_INT32 )); + EXPECT_STREQ("fixed64" , GetTypeNameForFieldType(FD::TYPE_FIXED64 )); + EXPECT_STREQ("fixed32" , GetTypeNameForFieldType(FD::TYPE_FIXED32 )); + EXPECT_STREQ("bool" , GetTypeNameForFieldType(FD::TYPE_BOOL )); + EXPECT_STREQ("string" , GetTypeNameForFieldType(FD::TYPE_STRING )); + EXPECT_STREQ("group" , GetTypeNameForFieldType(FD::TYPE_GROUP )); + EXPECT_STREQ("message" , GetTypeNameForFieldType(FD::TYPE_MESSAGE )); + EXPECT_STREQ("bytes" , GetTypeNameForFieldType(FD::TYPE_BYTES )); + EXPECT_STREQ("uint32" , GetTypeNameForFieldType(FD::TYPE_UINT32 )); + EXPECT_STREQ("enum" , GetTypeNameForFieldType(FD::TYPE_ENUM )); + EXPECT_STREQ("sfixed32", GetTypeNameForFieldType(FD::TYPE_SFIXED32)); + EXPECT_STREQ("sfixed64", GetTypeNameForFieldType(FD::TYPE_SFIXED64)); + EXPECT_STREQ("sint32" , GetTypeNameForFieldType(FD::TYPE_SINT32 )); + EXPECT_STREQ("sint64" , GetTypeNameForFieldType(FD::TYPE_SINT64 )); +} + TEST_F(MiscTest, CppTypes) { // Test that CPP types are assigned correctly. @@ -1576,6 +1620,31 @@ TEST_F(MiscTest, CppTypes) { EXPECT_EQ(FD::CPPTYPE_INT64 , GetCppTypeForFieldType(FD::TYPE_SINT64 )); } +TEST_F(MiscTest, CppTypeNames) { + // Test that correct CPP type names are returned. + + typedef FieldDescriptor FD; // avoid ugly line wrapping + + EXPECT_STREQ("double" , GetCppTypeNameForFieldType(FD::TYPE_DOUBLE )); + EXPECT_STREQ("float" , GetCppTypeNameForFieldType(FD::TYPE_FLOAT )); + EXPECT_STREQ("int64" , GetCppTypeNameForFieldType(FD::TYPE_INT64 )); + EXPECT_STREQ("uint64" , GetCppTypeNameForFieldType(FD::TYPE_UINT64 )); + EXPECT_STREQ("int32" , GetCppTypeNameForFieldType(FD::TYPE_INT32 )); + EXPECT_STREQ("uint64" , GetCppTypeNameForFieldType(FD::TYPE_FIXED64 )); + EXPECT_STREQ("uint32" , GetCppTypeNameForFieldType(FD::TYPE_FIXED32 )); + EXPECT_STREQ("bool" , GetCppTypeNameForFieldType(FD::TYPE_BOOL )); + EXPECT_STREQ("string" , GetCppTypeNameForFieldType(FD::TYPE_STRING )); + EXPECT_STREQ("message", GetCppTypeNameForFieldType(FD::TYPE_GROUP )); + EXPECT_STREQ("message", GetCppTypeNameForFieldType(FD::TYPE_MESSAGE )); + EXPECT_STREQ("string" , GetCppTypeNameForFieldType(FD::TYPE_BYTES )); + EXPECT_STREQ("uint32" , GetCppTypeNameForFieldType(FD::TYPE_UINT32 )); + EXPECT_STREQ("enum" , GetCppTypeNameForFieldType(FD::TYPE_ENUM )); + EXPECT_STREQ("int32" , GetCppTypeNameForFieldType(FD::TYPE_SFIXED32)); + EXPECT_STREQ("int64" , GetCppTypeNameForFieldType(FD::TYPE_SFIXED64)); + EXPECT_STREQ("int32" , GetCppTypeNameForFieldType(FD::TYPE_SINT32 )); + EXPECT_STREQ("int64" , GetCppTypeNameForFieldType(FD::TYPE_SINT64 )); +} + TEST_F(MiscTest, DefaultValues) { // Test that setting default values works. FileDescriptorProto file_proto; @@ -1739,13 +1808,31 @@ TEST_F(MiscTest, FieldOptions) { } // =================================================================== +enum DescriptorPoolMode { + NO_DATABASE, + FALLBACK_DATABASE +}; -class AllowUnknownDependenciesTest : public testing::Test { +class AllowUnknownDependenciesTest + : public testing::TestWithParam<DescriptorPoolMode> { protected: + DescriptorPoolMode mode() { + return GetParam(); + } + virtual void SetUp() { FileDescriptorProto foo_proto, bar_proto; - pool_.AllowUnknownDependencies(); + switch (mode()) { + case NO_DATABASE: + pool_.reset(new DescriptorPool); + break; + case FALLBACK_DATABASE: + pool_.reset(new DescriptorPool(&db_)); + break; + } + + pool_->AllowUnknownDependencies(); ASSERT_TRUE(TextFormat::ParseFromString( "name: 'foo.proto'" @@ -1776,13 +1863,13 @@ class AllowUnknownDependenciesTest : public testing::Test { &bar_proto)); // Collect pointers to stuff. - bar_file_ = pool_.BuildFile(bar_proto); + bar_file_ = BuildFile(bar_proto); ASSERT_TRUE(bar_file_ != NULL); ASSERT_EQ(1, bar_file_->message_type_count()); bar_type_ = bar_file_->message_type(0); - foo_file_ = pool_.BuildFile(foo_proto); + foo_file_ = BuildFile(foo_proto); ASSERT_TRUE(foo_file_ != NULL); ASSERT_EQ(1, foo_file_->message_type_count()); @@ -1794,6 +1881,20 @@ class AllowUnknownDependenciesTest : public testing::Test { qux_field_ = foo_type_->field(2); } + const FileDescriptor* BuildFile(const FileDescriptorProto& proto) { + switch (mode()) { + case NO_DATABASE: + return pool_->BuildFile(proto); + break; + case FALLBACK_DATABASE: { + EXPECT_TRUE(db_.Add(proto)); + return pool_->FindFileByName(proto.name()); + } + } + GOOGLE_LOG(FATAL) << "Can't get here."; + return NULL; + } + const FileDescriptor* bar_file_; const Descriptor* bar_type_; const FileDescriptor* foo_file_; @@ -1802,10 +1903,11 @@ class AllowUnknownDependenciesTest : public testing::Test { const FieldDescriptor* baz_field_; const FieldDescriptor* qux_field_; - DescriptorPool pool_; + SimpleDescriptorDatabase db_; // used if in FALLBACK_DATABASE mode. + scoped_ptr<DescriptorPool> pool_; }; -TEST_F(AllowUnknownDependenciesTest, PlaceholderFile) { +TEST_P(AllowUnknownDependenciesTest, PlaceholderFile) { ASSERT_EQ(2, foo_file_->dependency_count()); EXPECT_EQ(bar_file_, foo_file_->dependency(0)); @@ -1814,11 +1916,11 @@ TEST_F(AllowUnknownDependenciesTest, PlaceholderFile) { EXPECT_EQ(0, baz_file->message_type_count()); // Placeholder files should not be findable. - EXPECT_EQ(bar_file_, pool_.FindFileByName(bar_file_->name())); - EXPECT_TRUE(pool_.FindFileByName(baz_file->name()) == NULL); + EXPECT_EQ(bar_file_, pool_->FindFileByName(bar_file_->name())); + EXPECT_TRUE(pool_->FindFileByName(baz_file->name()) == NULL); } -TEST_F(AllowUnknownDependenciesTest, PlaceholderTypes) { +TEST_P(AllowUnknownDependenciesTest, PlaceholderTypes) { ASSERT_EQ(FieldDescriptor::TYPE_MESSAGE, bar_field_->type()); EXPECT_EQ(bar_type_, bar_field_->message_type()); @@ -1836,12 +1938,12 @@ TEST_F(AllowUnknownDependenciesTest, PlaceholderTypes) { EXPECT_EQ("corge.Qux.placeholder.proto", qux_type->file()->name()); // Placeholder types should not be findable. - EXPECT_EQ(bar_type_, pool_.FindMessageTypeByName(bar_type_->full_name())); - EXPECT_TRUE(pool_.FindMessageTypeByName(baz_type->full_name()) == NULL); - EXPECT_TRUE(pool_.FindEnumTypeByName(qux_type->full_name()) == NULL); + EXPECT_EQ(bar_type_, pool_->FindMessageTypeByName(bar_type_->full_name())); + EXPECT_TRUE(pool_->FindMessageTypeByName(baz_type->full_name()) == NULL); + EXPECT_TRUE(pool_->FindEnumTypeByName(qux_type->full_name()) == NULL); } -TEST_F(AllowUnknownDependenciesTest, CopyTo) { +TEST_P(AllowUnknownDependenciesTest, CopyTo) { // FieldDescriptor::CopyTo() should write non-fully-qualified type names // for placeholder types which were not originally fully-qualified. FieldDescriptorProto proto; @@ -1864,7 +1966,7 @@ TEST_F(AllowUnknownDependenciesTest, CopyTo) { EXPECT_EQ(FieldDescriptorProto::TYPE_ENUM, proto.type()); } -TEST_F(AllowUnknownDependenciesTest, CustomOptions) { +TEST_P(AllowUnknownDependenciesTest, CustomOptions) { // Qux should still have the uninterpreted option attached. ASSERT_EQ(1, qux_field_->options().uninterpreted_option_size()); const UninterpretedOption& option = @@ -1873,7 +1975,7 @@ TEST_F(AllowUnknownDependenciesTest, CustomOptions) { EXPECT_EQ("grault", option.name(0).name_part()); } -TEST_F(AllowUnknownDependenciesTest, UnknownExtendee) { +TEST_P(AllowUnknownDependenciesTest, UnknownExtendee) { // Test that we can extend an unknown type. This is slightly tricky because // it means that the placeholder type must have an extension range. @@ -1884,7 +1986,7 @@ TEST_F(AllowUnknownDependenciesTest, UnknownExtendee) { "extension { extendee: 'UnknownType' name:'some_extension' number:123" " label:LABEL_OPTIONAL type:TYPE_INT32 }", &extension_proto)); - const FileDescriptor* file = pool_.BuildFile(extension_proto); + const FileDescriptor* file = BuildFile(extension_proto); ASSERT_TRUE(file != NULL); @@ -1896,7 +1998,7 @@ TEST_F(AllowUnknownDependenciesTest, UnknownExtendee) { EXPECT_EQ(FieldDescriptor::kMaxNumber + 1, extendee->extension_range(0)->end); } -TEST_F(AllowUnknownDependenciesTest, CustomOption) { +TEST_P(AllowUnknownDependenciesTest, CustomOption) { // Test that we can use a custom option without having parsed // descriptor.proto. @@ -1937,7 +2039,7 @@ TEST_F(AllowUnknownDependenciesTest, CustomOption) { "}", &option_proto)); - const FileDescriptor* file = pool_.BuildFile(option_proto); + const FileDescriptor* file = BuildFile(option_proto); ASSERT_TRUE(file != NULL); // Verify that no extension options were set, but they were left as @@ -1949,6 +2051,81 @@ TEST_F(AllowUnknownDependenciesTest, CustomOption) { EXPECT_EQ(2, file->options().uninterpreted_option_size()); } +TEST_P(AllowUnknownDependenciesTest, + UndeclaredDependencyTriggersBuildOfDependency) { + // Crazy case: suppose foo.proto refers to a symbol without declaring the + // dependency that finds it. In the event that the pool is backed by a + // DescriptorDatabase, the pool will attempt to find the symbol in the + // database. If successful, it will build the undeclared dependency to verify + // that the file does indeed contain the symbol. If that file fails to build, + // then its descriptors must be rolled back. However, we still want foo.proto + // to build successfully, since we are allowing unknown dependencies. + + FileDescriptorProto undeclared_dep_proto; + // We make this file fail to build by giving it two fields with tag 1. + ASSERT_TRUE(TextFormat::ParseFromString( + "name: \"invalid_file_as_undeclared_dep.proto\" " + "package: \"undeclared\" " + "message_type: { " + " name: \"Quux\" " + " field { " + " name:'qux' number:1 label:LABEL_OPTIONAL type: TYPE_INT32 " + " }" + " field { " + " name:'quux' number:1 label:LABEL_OPTIONAL type: TYPE_INT64 " + " }" + "}", + &undeclared_dep_proto)); + // We can't use the BuildFile() helper because we don't actually want to build + // it into the descriptor pool in the fallback database case: it just needs to + // be sitting in the database so that it gets built during the building of + // test.proto below. + switch (mode()) { + case NO_DATABASE: { + ASSERT_TRUE(pool_->BuildFile(undeclared_dep_proto) == NULL); + break; + } + case FALLBACK_DATABASE: { + ASSERT_TRUE(db_.Add(undeclared_dep_proto)); + } + } + + FileDescriptorProto test_proto; + ASSERT_TRUE(TextFormat::ParseFromString( + "name: \"test.proto\" " + "message_type: { " + " name: \"Corge\" " + " field { " + " name:'quux' number:1 label: LABEL_OPTIONAL " + " type_name:'undeclared.Quux' type: TYPE_MESSAGE " + " }" + "}", + &test_proto)); + + const FileDescriptor* file = BuildFile(test_proto); + ASSERT_TRUE(file != NULL); + GOOGLE_LOG(INFO) << file->DebugString(); + + EXPECT_EQ(0, file->dependency_count()); + ASSERT_EQ(1, file->message_type_count()); + const Descriptor* corge_desc = file->message_type(0); + ASSERT_EQ("Corge", corge_desc->name()); + ASSERT_EQ(1, corge_desc->field_count()); + + const FieldDescriptor* quux_field = corge_desc->field(0); + ASSERT_EQ(FieldDescriptor::TYPE_MESSAGE, quux_field->type()); + ASSERT_EQ("Quux", quux_field->message_type()->name()); + ASSERT_EQ("undeclared.Quux", quux_field->message_type()->full_name()); + EXPECT_EQ("undeclared.Quux.placeholder.proto", + quux_field->message_type()->file()->name()); + // The place holder type should not be findable. + ASSERT_TRUE(pool_->FindMessageTypeByName("undeclared.Quux") == NULL); +} + +INSTANTIATE_TEST_CASE_P(DatabaseSource, + AllowUnknownDependenciesTest, + testing::Values(NO_DATABASE, FALLBACK_DATABASE)); + // =================================================================== TEST(CustomOptions, OptionLocations) { @@ -2451,6 +2628,15 @@ TEST_F(ValidationErrorTest, UnknownDependency) { "bar.proto: bar.proto: OTHER: Import \"foo.proto\" has not been loaded.\n"); } +TEST_F(ValidationErrorTest, InvalidPublicDependencyIndex) { + BuildFile("name: \"foo.proto\""); + BuildFileWithErrors( + "name: \"bar.proto\" " + "dependency: \"foo.proto\" " + "public_dependency: 1", + "bar.proto: bar.proto: OTHER: Invalid public dependency index.\n"); +} + TEST_F(ValidationErrorTest, ForeignUnimportedPackageNoCrash) { // Used to crash: If we depend on a non-existent file and then refer to a // package defined in a file that we didn't import, and that package is @@ -2829,6 +3015,164 @@ TEST_F(ValidationErrorTest, FieldTypeDefinedInUndeclaredDependency) { "necessary import.\n"); } +TEST_F(ValidationErrorTest, FieldTypeDefinedInIndirectDependency) { + // Test for hidden dependencies. + // + // // bar.proto + // message Bar{} + // + // // forward.proto + // import "bar.proto" + // + // // foo.proto + // import "forward.proto" + // message Foo { + // optional Bar foo = 1; // Error, needs to import bar.proto explicitly. + // } + // + BuildFile( + "name: \"bar.proto\" " + "message_type { name: \"Bar\" }"); + + BuildFile( + "name: \"forward.proto\"" + "dependency: \"bar.proto\""); + + BuildFileWithErrors( + "name: \"foo.proto\" " + "dependency: \"forward.proto\" " + "message_type {" + " name: \"Foo\"" + " field { name:\"foo\" number:1 label:LABEL_OPTIONAL type_name:\"Bar\" }" + "}", + "foo.proto: Foo.foo: TYPE: \"Bar\" seems to be defined in \"bar.proto\", " + "which is not imported by \"foo.proto\". To use it here, please add the " + "necessary import.\n"); +} + +TEST_F(ValidationErrorTest, FieldTypeDefinedInPublicDependency) { + // Test for public dependencies. + // + // // bar.proto + // message Bar{} + // + // // forward.proto + // import public "bar.proto" + // + // // foo.proto + // import "forward.proto" + // message Foo { + // optional Bar foo = 1; // Correct. "bar.proto" is public imported into + // // forward.proto, so when "foo.proto" imports + // // "forward.proto", it imports "bar.proto" too. + // } + // + BuildFile( + "name: \"bar.proto\" " + "message_type { name: \"Bar\" }"); + + BuildFile( + "name: \"forward.proto\"" + "dependency: \"bar.proto\" " + "public_dependency: 0"); + + BuildFile( + "name: \"foo.proto\" " + "dependency: \"forward.proto\" " + "message_type {" + " name: \"Foo\"" + " field { name:\"foo\" number:1 label:LABEL_OPTIONAL type_name:\"Bar\" }" + "}"); +} + +TEST_F(ValidationErrorTest, FieldTypeDefinedInTransitivePublicDependency) { + // Test for public dependencies. + // + // // bar.proto + // message Bar{} + // + // // forward.proto + // import public "bar.proto" + // + // // forward2.proto + // import public "forward.proto" + // + // // foo.proto + // import "forward2.proto" + // message Foo { + // optional Bar foo = 1; // Correct, public imports are transitive. + // } + // + BuildFile( + "name: \"bar.proto\" " + "message_type { name: \"Bar\" }"); + + BuildFile( + "name: \"forward.proto\"" + "dependency: \"bar.proto\" " + "public_dependency: 0"); + + BuildFile( + "name: \"forward2.proto\"" + "dependency: \"forward.proto\" " + "public_dependency: 0"); + + BuildFile( + "name: \"foo.proto\" " + "dependency: \"forward2.proto\" " + "message_type {" + " name: \"Foo\"" + " field { name:\"foo\" number:1 label:LABEL_OPTIONAL type_name:\"Bar\" }" + "}"); +} + +TEST_F(ValidationErrorTest, + FieldTypeDefinedInPrivateDependencyOfPublicDependency) { + // Test for public dependencies. + // + // // bar.proto + // message Bar{} + // + // // forward.proto + // import "bar.proto" + // + // // forward2.proto + // import public "forward.proto" + // + // // foo.proto + // import "forward2.proto" + // message Foo { + // optional Bar foo = 1; // Error, the "bar.proto" is not public imported + // // into "forward.proto", so will not be imported + // // into either "forward2.proto" or "foo.proto". + // } + // + BuildFile( + "name: \"bar.proto\" " + "message_type { name: \"Bar\" }"); + + BuildFile( + "name: \"forward.proto\"" + "dependency: \"bar.proto\""); + + BuildFile( + "name: \"forward2.proto\"" + "dependency: \"forward.proto\" " + "public_dependency: 0"); + + BuildFileWithErrors( + "name: \"foo.proto\" " + "dependency: \"forward2.proto\" " + "message_type {" + " name: \"Foo\"" + " field { name:\"foo\" number:1 label:LABEL_OPTIONAL type_name:\"Bar\" }" + "}", + "foo.proto: Foo.foo: TYPE: \"Bar\" seems to be defined in \"bar.proto\", " + "which is not imported by \"foo.proto\". To use it here, please add the " + "necessary import.\n"); +} + + TEST_F(ValidationErrorTest, SearchMostLocalFirst) { // The following should produce an error that Bar.Baz is not defined: // message Bar { message Baz {} } @@ -3031,7 +3375,8 @@ TEST_F(ValidationErrorTest, InputTypeNotDefined) { " method { name: \"A\" input_type: \"Bar\" output_type: \"Foo\" }" "}", - "foo.proto: TestService.A: INPUT_TYPE: \"Bar\" is not defined.\n"); + "foo.proto: TestService.A: INPUT_TYPE: \"Bar\" is not defined.\n" + ); } TEST_F(ValidationErrorTest, InputTypeNotAMessage) { @@ -3044,7 +3389,8 @@ TEST_F(ValidationErrorTest, InputTypeNotAMessage) { " method { name: \"A\" input_type: \"Bar\" output_type: \"Foo\" }" "}", - "foo.proto: TestService.A: INPUT_TYPE: \"Bar\" is not a message type.\n"); + "foo.proto: TestService.A: INPUT_TYPE: \"Bar\" is not a message type.\n" + ); } TEST_F(ValidationErrorTest, OutputTypeNotDefined) { @@ -3056,7 +3402,8 @@ TEST_F(ValidationErrorTest, OutputTypeNotDefined) { " method { name: \"A\" input_type: \"Foo\" output_type: \"Bar\" }" "}", - "foo.proto: TestService.A: OUTPUT_TYPE: \"Bar\" is not defined.\n"); + "foo.proto: TestService.A: OUTPUT_TYPE: \"Bar\" is not defined.\n" + ); } TEST_F(ValidationErrorTest, OutputTypeNotAMessage) { @@ -3069,9 +3416,11 @@ TEST_F(ValidationErrorTest, OutputTypeNotAMessage) { " method { name: \"A\" input_type: \"Foo\" output_type: \"Bar\" }" "}", - "foo.proto: TestService.A: OUTPUT_TYPE: \"Bar\" is not a message type.\n"); + "foo.proto: TestService.A: OUTPUT_TYPE: \"Bar\" is not a message type.\n" + ); } + TEST_F(ValidationErrorTest, IllegalPackedField) { BuildFileWithErrors( "name: \"foo.proto\" " @@ -3592,7 +3941,8 @@ TEST_F(ValidationErrorTest, RollbackAfterError) { " }" "}", - "foo.proto: TestService.Baz: INPUT_TYPE: \"NoSuchType\" is not defined.\n"); + "foo.proto: TestService.Baz: INPUT_TYPE: \"NoSuchType\" is not defined.\n" + ); // Make sure that if we build the same file again with the error fixed, // it works. If the above rollback was incomplete, then some symbols will @@ -3641,6 +3991,19 @@ TEST_F(ValidationErrorTest, ErrorsReportedToLogError) { EXPECT_EQ(" Foo: \"Foo\" is already defined.", errors[1]); } +TEST_F(ValidationErrorTest, DisallowEnumAlias) { + BuildFileWithErrors( + "name: \"foo.proto\" " + "enum_type {" + " name: \"Bar\"" + " value { name:\"ENUM_A\" number:0 }" + " value { name:\"ENUM_B\" number:0 }" + " options { allow_alias: false }" + "}", + "foo.proto: Bar: NUMBER: " + "\"ENUM_B\" uses the same enum value as \"ENUM_A\"\n"); +} + // =================================================================== // DescriptorDatabase @@ -3659,16 +4022,23 @@ class DatabaseBackedPoolTest : public testing::Test { virtual void SetUp() { AddToDatabase(&database_, - "name: \"foo.proto\" " - "message_type { name:\"Foo\" extension_range { start: 1 end: 100 } } " - "enum_type { name:\"TestEnum\" value { name:\"DUMMY\" number:0 } } " - "service { name:\"TestService\" } "); + "name: 'foo.proto' " + "message_type { name:'Foo' extension_range { start: 1 end: 100 } } " + "enum_type { name:'TestEnum' value { name:'DUMMY' number:0 } } " + "service { name:'TestService' } "); AddToDatabase(&database_, - "name: \"bar.proto\" " - "dependency: \"foo.proto\" " - "message_type { name:\"Bar\" } " - "extension { name:\"foo_ext\" extendee: \".Foo\" number:5 " + "name: 'bar.proto' " + "dependency: 'foo.proto' " + "message_type { name:'Bar' } " + "extension { name:'foo_ext' extendee: '.Foo' number:5 " " label:LABEL_OPTIONAL type:TYPE_INT32 } "); + // Baz has an undeclared dependency on Foo. + AddToDatabase(&database_, + "name: 'baz.proto' " + "message_type { " + " name:'Baz' " + " field { name:'foo' number:1 label:LABEL_OPTIONAL type_name:'Foo' } " + "}"); } // We can't inject a file containing errors into a DescriptorPool, so we @@ -3907,6 +4277,33 @@ TEST_F(DatabaseBackedPoolTest, ErrorWithErrorCollector) { error_collector.text_); } +TEST_F(DatabaseBackedPoolTest, UndeclaredDependencyOnUnbuiltType) { + // Check that we find and report undeclared dependencies on types that exist + // in the descriptor database but that have not not been built yet. + MockErrorCollector error_collector; + DescriptorPool pool(&database_, &error_collector); + EXPECT_TRUE(pool.FindMessageTypeByName("Baz") == NULL); + EXPECT_EQ( + "baz.proto: Baz.foo: TYPE: \"Foo\" seems to be defined in \"foo.proto\", " + "which is not imported by \"baz.proto\". To use it here, please add " + "the necessary import.\n", + error_collector.text_); +} + +TEST_F(DatabaseBackedPoolTest, RollbackAfterError) { + // Make sure that all traces of bad types are removed from the pool. This used + // to be b/4529436, due to the fact that a symbol resolution failure could + // potentially cause another file to be recursively built, which would trigger + // a checkpoint _past_ possibly invalid symbols. + // Baz is defined in the database, but the file is invalid because it is + // missing a necessary import. + DescriptorPool pool(&database_); + EXPECT_TRUE(pool.FindMessageTypeByName("Baz") == NULL); + // Make sure that searching again for the file or the type fails. + EXPECT_TRUE(pool.FindFileByName("baz.proto") == NULL); + EXPECT_TRUE(pool.FindMessageTypeByName("Baz") == NULL); +} + TEST_F(DatabaseBackedPoolTest, UnittestProto) { // Try to load all of unittest.proto from a DescriptorDatabase. This should // thoroughly test all paths through DescriptorBuilder to insure that there @@ -3963,6 +4360,16 @@ TEST_F(DatabaseBackedPoolTest, DoesntRetryDbUnnecessarily) { EXPECT_TRUE(file->FindEnumValueByName("NO_SUCH_VALUE") == NULL); EXPECT_TRUE(file->FindServiceByName("NO_SUCH_VALUE") == NULL); EXPECT_TRUE(file->FindExtensionByName("no_such_extension") == NULL); + + EXPECT_TRUE(pool.FindFileContainingSymbol("Foo.no.such.field") == NULL); + EXPECT_TRUE(pool.FindFileContainingSymbol("Foo.no_such_field") == NULL); + EXPECT_TRUE(pool.FindMessageTypeByName("Foo.NoSuchMessageType") == NULL); + EXPECT_TRUE(pool.FindFieldByName("Foo.no_such_field") == NULL); + EXPECT_TRUE(pool.FindExtensionByName("Foo.no_such_extension") == NULL); + EXPECT_TRUE(pool.FindEnumTypeByName("Foo.NoSuchEnumType") == NULL); + EXPECT_TRUE(pool.FindEnumValueByName("Foo.NO_SUCH_VALUE") == NULL); + EXPECT_TRUE(pool.FindMethodByName("TestService.NoSuchMethod") == NULL); + EXPECT_EQ(0, call_counter.call_count_); } @@ -4028,6 +4435,219 @@ TEST_F(DatabaseBackedPoolTest, DoesntFallbackOnWrongType) { // =================================================================== +class AbortingErrorCollector : public DescriptorPool::ErrorCollector { + public: + AbortingErrorCollector() {} + + virtual void AddError( + const string &filename, + const string &element_name, + const Message *message, + ErrorLocation location, + const string &error_message) { + GOOGLE_LOG(FATAL) << "AddError() called unexpectedly: " << filename << ": " + << error_message; + } + private: + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(AbortingErrorCollector); +}; + +// A source tree containing only one file. +class SingletonSourceTree : public compiler::SourceTree { + public: + SingletonSourceTree(const string& filename, const string& contents) + : filename_(filename), contents_(contents) {} + + virtual io::ZeroCopyInputStream* Open(const string& filename) { + return filename == filename_ ? + new io::ArrayInputStream(contents_.data(), contents_.size()) : NULL; + } + + private: + const string filename_; + const string contents_; + + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(SingletonSourceTree); +}; + +const char *const kSourceLocationTestInput = + "syntax = \"proto2\";\n" + "message A {\n" + " optional int32 a = 1;\n" + " message B {\n" + " required double b = 1;\n" + " }\n" + "}\n" + "enum Indecision {\n" + " YES = 1;\n" + " NO = 2;\n" + " MAYBE = 3;\n" + "}\n" + "service S {\n" + " rpc Method(A) returns (A.B);\n" + // Put an empty line here to make the source location range match. + "\n" + "}\n"; + +class SourceLocationTest : public testing::Test { + public: + SourceLocationTest() + : source_tree_("/test/test.proto", kSourceLocationTestInput), + db_(&source_tree_), + pool_(&db_, &collector_) {} + + static string PrintSourceLocation(const SourceLocation &loc) { + return strings::Substitute("$0:$1-$2:$3", + 1 + loc.start_line, + 1 + loc.start_column, + 1 + loc.end_line, + 1 + loc.end_column); + } + + private: + AbortingErrorCollector collector_; + SingletonSourceTree source_tree_; + compiler::SourceTreeDescriptorDatabase db_; + + protected: + DescriptorPool pool_; +}; + +// TODO(adonovan): implement support for option fields and for +// subparts of declarations. + +TEST_F(SourceLocationTest, GetSourceLocation) { + SourceLocation loc; + + const FileDescriptor *file_desc = + GOOGLE_CHECK_NOTNULL(pool_.FindFileByName("/test/test.proto")); + + const Descriptor *a_desc = file_desc->FindMessageTypeByName("A"); + EXPECT_TRUE(a_desc->GetSourceLocation(&loc)); + EXPECT_EQ("2:1-7:2", PrintSourceLocation(loc)); + + const Descriptor *a_b_desc = a_desc->FindNestedTypeByName("B"); + EXPECT_TRUE(a_b_desc->GetSourceLocation(&loc)); + EXPECT_EQ("4:3-6:4", PrintSourceLocation(loc)); + + const EnumDescriptor *e_desc = file_desc->FindEnumTypeByName("Indecision"); + EXPECT_TRUE(e_desc->GetSourceLocation(&loc)); + EXPECT_EQ("8:1-12:2", PrintSourceLocation(loc)); + + const EnumValueDescriptor *yes_desc = e_desc->FindValueByName("YES"); + EXPECT_TRUE(yes_desc->GetSourceLocation(&loc)); + EXPECT_EQ("9:3-9:13", PrintSourceLocation(loc)); + + const ServiceDescriptor *s_desc = file_desc->FindServiceByName("S"); + EXPECT_TRUE(s_desc->GetSourceLocation(&loc)); + EXPECT_EQ("13:1-16:2", PrintSourceLocation(loc)); + + const MethodDescriptor *m_desc = s_desc->FindMethodByName("Method"); + EXPECT_TRUE(m_desc->GetSourceLocation(&loc)); + EXPECT_EQ("14:3-14:31", PrintSourceLocation(loc)); + +} + +// Missing SourceCodeInfo doesn't cause crash: +TEST_F(SourceLocationTest, GetSourceLocation_MissingSourceCodeInfo) { + SourceLocation loc; + + const FileDescriptor *file_desc = + GOOGLE_CHECK_NOTNULL(pool_.FindFileByName("/test/test.proto")); + + FileDescriptorProto proto; + file_desc->CopyTo(&proto); // Note, this discards the SourceCodeInfo. + EXPECT_FALSE(proto.has_source_code_info()); + + DescriptorPool bad1_pool(&pool_); + const FileDescriptor* bad1_file_desc = + GOOGLE_CHECK_NOTNULL(bad1_pool.BuildFile(proto)); + const Descriptor *bad1_a_desc = bad1_file_desc->FindMessageTypeByName("A"); + EXPECT_FALSE(bad1_a_desc->GetSourceLocation(&loc)); +} + +// Corrupt SourceCodeInfo doesn't cause crash: +TEST_F(SourceLocationTest, GetSourceLocation_BogusSourceCodeInfo) { + SourceLocation loc; + + const FileDescriptor *file_desc = + GOOGLE_CHECK_NOTNULL(pool_.FindFileByName("/test/test.proto")); + + FileDescriptorProto proto; + file_desc->CopyTo(&proto); // Note, this discards the SourceCodeInfo. + EXPECT_FALSE(proto.has_source_code_info()); + SourceCodeInfo_Location *loc_msg = + proto.mutable_source_code_info()->add_location(); + loc_msg->add_path(1); + loc_msg->add_path(2); + loc_msg->add_path(3); + loc_msg->add_span(4); + loc_msg->add_span(5); + loc_msg->add_span(6); + + DescriptorPool bad2_pool(&pool_); + const FileDescriptor* bad2_file_desc = + GOOGLE_CHECK_NOTNULL(bad2_pool.BuildFile(proto)); + const Descriptor *bad2_a_desc = bad2_file_desc->FindMessageTypeByName("A"); + EXPECT_FALSE(bad2_a_desc->GetSourceLocation(&loc)); +} + +// =================================================================== + +const char* const kCopySourceCodeInfoToTestInput = + "syntax = \"proto2\";\n" + "message Foo {}\n"; + +// Required since source code information is not preserved by +// FileDescriptorTest. +class CopySourceCodeInfoToTest : public testing::Test { + public: + CopySourceCodeInfoToTest() + : source_tree_("/test/test.proto", kCopySourceCodeInfoToTestInput), + db_(&source_tree_), + pool_(&db_, &collector_) {} + + private: + AbortingErrorCollector collector_; + SingletonSourceTree source_tree_; + compiler::SourceTreeDescriptorDatabase db_; + + protected: + DescriptorPool pool_; +}; + +TEST_F(CopySourceCodeInfoToTest, CopyTo_DoesNotCopySourceCodeInfo) { + const FileDescriptor* file_desc = + GOOGLE_CHECK_NOTNULL(pool_.FindFileByName("/test/test.proto")); + FileDescriptorProto file_desc_proto; + ASSERT_FALSE(file_desc_proto.has_source_code_info()); + + file_desc->CopyTo(&file_desc_proto); + EXPECT_FALSE(file_desc_proto.has_source_code_info()); +} + +TEST_F(CopySourceCodeInfoToTest, CopySourceCodeInfoTo) { + const FileDescriptor* file_desc = + GOOGLE_CHECK_NOTNULL(pool_.FindFileByName("/test/test.proto")); + FileDescriptorProto file_desc_proto; + ASSERT_FALSE(file_desc_proto.has_source_code_info()); + + file_desc->CopySourceCodeInfoTo(&file_desc_proto); + const SourceCodeInfo& info = file_desc_proto.source_code_info(); + ASSERT_EQ(3, info.location_size()); + // Get the Foo message location + const SourceCodeInfo_Location& foo_location = info.location(1); + ASSERT_EQ(2, foo_location.path_size()); + EXPECT_EQ(FileDescriptorProto::kMessageTypeFieldNumber, foo_location.path(0)); + EXPECT_EQ(0, foo_location.path(1)); // Foo is the first message defined + ASSERT_EQ(3, foo_location.span_size()); // Foo spans one line + EXPECT_EQ(1, foo_location.span(0)); // Foo is declared on line 1 + EXPECT_EQ(0, foo_location.span(1)); // Foo starts at column 0 + EXPECT_EQ(14, foo_location.span(2)); // Foo ends on column 14 +} + +// =================================================================== + } // namespace descriptor_unittest } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/dynamic_message.cc b/third_party/protobuf/src/google/protobuf/dynamic_message.cc index c711a2d..e4bebb2 100644 --- a/third_party/protobuf/src/google/protobuf/dynamic_message.cc +++ b/third_party/protobuf/src/google/protobuf/dynamic_message.cc @@ -123,7 +123,9 @@ int FieldSpaceUsed(const FieldDescriptor* field) { case FD::CPPTYPE_FLOAT : return sizeof(float ); case FD::CPPTYPE_BOOL : return sizeof(bool ); case FD::CPPTYPE_ENUM : return sizeof(int ); - case FD::CPPTYPE_MESSAGE: return sizeof(Message*); + + case FD::CPPTYPE_MESSAGE: + return sizeof(Message*); case FD::CPPTYPE_STRING: switch (field->options().ctype()) { @@ -178,7 +180,17 @@ class DynamicMessage : public Message { // important (the prototype must be deleted *before* the offsets). scoped_array<int> offsets; scoped_ptr<const GeneratedMessageReflection> reflection; - scoped_ptr<const DynamicMessage> prototype; + // Don't use a scoped_ptr to hold the prototype: the destructor for + // DynamicMessage needs to know whether it is the prototype, and does so by + // looking back at this field. This would assume details about the + // implementation of scoped_ptr. + const DynamicMessage* prototype; + + TypeInfo() : prototype(NULL) {} + + ~TypeInfo() { + delete prototype; + } }; DynamicMessage(const TypeInfo* type_info); @@ -196,6 +208,14 @@ class DynamicMessage : public Message { Metadata GetMetadata() const; + // We actually allocate more memory than sizeof(*this) when this + // class's memory is allocated via the global operator new. Thus, we need to + // manually call the global operator delete. Calling the destructor is taken + // care of for us. + static void operator delete(void* ptr) { + ::operator delete(ptr); + } + private: GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(DynamicMessage); @@ -368,11 +388,12 @@ DynamicMessage::~DynamicMessage() { break; } } - } else if ((field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) && - !is_prototype()) { - Message* message = *reinterpret_cast<Message**>(field_ptr); - if (message != NULL) { - delete message; + } else if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { + if (!is_prototype()) { + Message* message = *reinterpret_cast<Message**>(field_ptr); + if (message != NULL) { + delete message; + } } } } @@ -403,7 +424,7 @@ void DynamicMessage::CrossLinkPrototypes() { } Message* DynamicMessage::New() const { - void* new_base = reinterpret_cast<uint8*>(operator new(type_info_->size)); + void* new_base = operator new(type_info_->size); memset(new_base, 0, type_info_->size); return new(new_base) DynamicMessage(type_info_); } @@ -465,7 +486,7 @@ const Message* DynamicMessageFactory::GetPrototypeNoLock( const DynamicMessage::TypeInfo** target = &prototypes_->map_[type]; if (*target != NULL) { // Already exists. - return (*target)->prototype.get(); + return (*target)->prototype; } DynamicMessage::TypeInfo* type_info = new DynamicMessage::TypeInfo; @@ -533,13 +554,13 @@ const Message* DynamicMessageFactory::GetPrototypeNoLock( void* base = operator new(size); memset(base, 0, size); DynamicMessage* prototype = new(base) DynamicMessage(type_info); - type_info->prototype.reset(prototype); + type_info->prototype = prototype; // Construct the reflection object. type_info->reflection.reset( new GeneratedMessageReflection( type_info->type, - type_info->prototype.get(), + type_info->prototype, type_info->offsets.get(), type_info->has_bits_offset, type_info->unknown_fields_offset, diff --git a/third_party/protobuf/src/google/protobuf/dynamic_message.h b/third_party/protobuf/src/google/protobuf/dynamic_message.h index 81dd2c6..b3d1e5d 100644 --- a/third_party/protobuf/src/google/protobuf/dynamic_message.h +++ b/third_party/protobuf/src/google/protobuf/dynamic_message.h @@ -68,7 +68,7 @@ class DescriptorPool; // descriptor.h class LIBPROTOBUF_EXPORT DynamicMessageFactory : public MessageFactory { public: // Construct a DynamicMessageFactory that will search for extensions in - // the DescriptorPool in which the exendee is defined. + // the DescriptorPool in which the extendee is defined. DynamicMessageFactory(); // Construct a DynamicMessageFactory that will search for extensions in @@ -102,7 +102,7 @@ class LIBPROTOBUF_EXPORT DynamicMessageFactory : public MessageFactory { // object. The returned object remains property of the factory and will // be destroyed when the factory is destroyed. Also, any objects created // by calling the prototype's New() method share some data with the - // prototype, so these must be destoyed before the DynamicMessageFactory + // prototype, so these must be destroyed before the DynamicMessageFactory // is destroyed. // // The given descriptor must outlive the returned message, and hence must diff --git a/third_party/protobuf/src/google/protobuf/dynamic_message_unittest.cc b/third_party/protobuf/src/google/protobuf/dynamic_message_unittest.cc index 41b89ab..e461597 100644 --- a/third_party/protobuf/src/google/protobuf/dynamic_message_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/dynamic_message_unittest.cc @@ -73,11 +73,15 @@ class DynamicMessageTest : public testing::Test { // unittest_import.proto. FileDescriptorProto unittest_file; FileDescriptorProto unittest_import_file; + FileDescriptorProto unittest_import_public_file; unittest::TestAllTypes::descriptor()->file()->CopyTo(&unittest_file); unittest_import::ImportMessage::descriptor()->file()->CopyTo( &unittest_import_file); + unittest_import::PublicImportMessage::descriptor()->file()->CopyTo( + &unittest_import_public_file); + ASSERT_TRUE(pool_.BuildFile(unittest_import_public_file) != NULL); ASSERT_TRUE(pool_.BuildFile(unittest_import_file) != NULL); ASSERT_TRUE(pool_.BuildFile(unittest_file) != NULL); diff --git a/third_party/protobuf/src/google/protobuf/extension_set.cc b/third_party/protobuf/src/google/protobuf/extension_set.cc index dbb0915..418e068 100644 --- a/third_party/protobuf/src/google/protobuf/extension_set.cc +++ b/third_party/protobuf/src/google/protobuf/extension_set.cc @@ -71,7 +71,7 @@ void DeleteRegistry() { void InitRegistry() { registry_ = new ExtensionRegistry; - internal::OnShutdown(&DeleteRegistry); + OnShutdown(&DeleteRegistry); } // This function is only called at startup, so there is no need for thread- @@ -180,6 +180,17 @@ bool ExtensionSet::Has(int number) const { return !iter->second.is_cleared; } +int ExtensionSet::NumExtensions() const { + int result = 0; + for (map<int, Extension>::const_iterator iter = extensions_.begin(); + iter != extensions_.end(); ++iter) { + if (!iter->second.is_cleared) { + ++result; + } + } + return result; +} + int ExtensionSet::ExtensionSize(int number) const { map<int, Extension>::const_iterator iter = extensions_.find(number); if (iter == extensions_.end()) return false; @@ -293,6 +304,14 @@ PRIMITIVE_ACCESSORS( BOOL, bool, Bool) #undef PRIMITIVE_ACCESSORS +void* ExtensionSet::MutableRawRepeatedField(int number) { + // We assume that all the RepeatedField<>* pointers have the same + // size and alignment within the anonymous union in Extension. + map<int, Extension>::const_iterator iter = extensions_.find(number); + GOOGLE_CHECK(iter != extensions_.end()) << "no extension numbered " << number; + return iter->second.repeated_int32_value; +} + // ------------------------------------------------------------------- // Enums @@ -422,7 +441,11 @@ const MessageLite& ExtensionSet::GetMessage( return default_value; } else { GOOGLE_DCHECK_TYPE(iter->second, OPTIONAL, MESSAGE); - return *iter->second.message_value; + if (iter->second.is_lazy) { + return iter->second.lazymessage_value->GetMessage(default_value); + } else { + return *iter->second.message_value; + } } } @@ -439,12 +462,19 @@ MessageLite* ExtensionSet::MutableMessage(int number, FieldType type, extension->type = type; GOOGLE_DCHECK_EQ(cpp_type(extension->type), WireFormatLite::CPPTYPE_MESSAGE); extension->is_repeated = false; + extension->is_lazy = false; extension->message_value = prototype.New(); + extension->is_cleared = false; + return extension->message_value; } else { GOOGLE_DCHECK_TYPE(*extension, OPTIONAL, MESSAGE); + extension->is_cleared = false; + if (extension->is_lazy) { + return extension->lazymessage_value->MutableMessage(prototype); + } else { + return extension->message_value; + } } - extension->is_cleared = false; - return extension->message_value; } // Defined in extension_set_heavy.cc. @@ -452,6 +482,56 @@ MessageLite* ExtensionSet::MutableMessage(int number, FieldType type, // const Descriptor* message_type, // MessageFactory* factory) +void ExtensionSet::SetAllocatedMessage(int number, FieldType type, + const FieldDescriptor* descriptor, + MessageLite* message) { + if (message == NULL) { + ClearExtension(number); + return; + } + Extension* extension; + if (MaybeNewExtension(number, descriptor, &extension)) { + extension->type = type; + GOOGLE_DCHECK_EQ(cpp_type(extension->type), WireFormatLite::CPPTYPE_MESSAGE); + extension->is_repeated = false; + extension->is_lazy = false; + extension->message_value = message; + } else { + GOOGLE_DCHECK_TYPE(*extension, OPTIONAL, MESSAGE); + if (extension->is_lazy) { + extension->lazymessage_value->SetAllocatedMessage(message); + } else { + delete extension->message_value; + extension->message_value = message; + } + } + extension->is_cleared = false; +} + +MessageLite* ExtensionSet::ReleaseMessage(int number, + const MessageLite& prototype) { + map<int, Extension>::iterator iter = extensions_.find(number); + if (iter == extensions_.end()) { + // Not present. Return NULL. + return NULL; + } else { + GOOGLE_DCHECK_TYPE(iter->second, OPTIONAL, MESSAGE); + MessageLite* ret = NULL; + if (iter->second.is_lazy) { + ret = iter->second.lazymessage_value->ReleaseMessage(prototype); + delete iter->second.lazymessage_value; + } else { + ret = iter->second.message_value; + } + extensions_.erase(number); + return ret; + } +} + +// Defined in extension_set_heavy.cc. +// MessageLite* ExtensionSet::ReleaseMessage(const FieldDescriptor* descriptor, +// MessageFactory* factory); + const MessageLite& ExtensionSet::GetRepeatedMessage( int number, int index) const { map<int, Extension>::const_iterator iter = extensions_.find(number); @@ -484,7 +564,7 @@ MessageLite* ExtensionSet::AddMessage(int number, FieldType type, // RepeatedPtrField<MessageLite> does not know how to Add() since it cannot // allocate an abstract object, so we have to be tricky. MessageLite* result = extension->repeated_message_value - ->AddFromCleared<internal::GenericTypeHandler<MessageLite> >(); + ->AddFromCleared<GenericTypeHandler<MessageLite> >(); if (result == NULL) { result = prototype.New(); extension->repeated_message_value->AddAllocated(result); @@ -540,6 +620,16 @@ void ExtensionSet::RemoveLast(int number) { } } +MessageLite* ExtensionSet::ReleaseLast(int number) { + map<int, Extension>::iterator iter = extensions_.find(number); + GOOGLE_CHECK(iter != extensions_.end()) << "Index out-of-bounds (field is empty)."; + + Extension* extension = &iter->second; + GOOGLE_DCHECK(extension->is_repeated); + GOOGLE_DCHECK(cpp_type(extension->type) == WireFormatLite::CPPTYPE_MESSAGE); + return extension->repeated_message_value->ReleaseLast(); +} + void ExtensionSet::SwapElements(int number, int index1, int index2) { map<int, Extension>::iterator iter = extensions_.find(number); GOOGLE_CHECK(iter != extensions_.end()) << "Index out-of-bounds (field is empty)."; @@ -602,9 +692,11 @@ void ExtensionSet::MergeFrom(const ExtensionSet& other) { if (is_new) { // Extension did not already exist in set. extension->type = other_extension.type; + extension->is_packed = other_extension.is_packed; extension->is_repeated = true; } else { GOOGLE_DCHECK_EQ(extension->type, other_extension.type); + GOOGLE_DCHECK_EQ(extension->is_packed, other_extension.is_packed); GOOGLE_DCHECK(extension->is_repeated); } @@ -675,12 +767,55 @@ void ExtensionSet::MergeFrom(const ExtensionSet& other) { *other_extension.string_value, other_extension.descriptor); break; - case WireFormatLite::CPPTYPE_MESSAGE: - MutableMessage(iter->first, other_extension.type, - *other_extension.message_value, - other_extension.descriptor) - ->CheckTypeAndMergeFrom(*other_extension.message_value); + case WireFormatLite::CPPTYPE_MESSAGE: { + Extension* extension; + bool is_new = MaybeNewExtension(iter->first, + other_extension.descriptor, + &extension); + if (is_new) { + extension->type = other_extension.type; + extension->is_packed = other_extension.is_packed; + extension->is_repeated = false; + if (other_extension.is_lazy) { + extension->is_lazy = true; + extension->lazymessage_value = + other_extension.lazymessage_value->New(); + extension->lazymessage_value->MergeFrom( + *other_extension.lazymessage_value); + } else { + extension->is_lazy = false; + extension->message_value = + other_extension.message_value->New(); + extension->message_value->CheckTypeAndMergeFrom( + *other_extension.message_value); + } + } else { + GOOGLE_DCHECK_EQ(extension->type, other_extension.type); + GOOGLE_DCHECK_EQ(extension->is_packed,other_extension.is_packed); + GOOGLE_DCHECK(!extension->is_repeated); + if (other_extension.is_lazy) { + if (extension->is_lazy) { + extension->lazymessage_value->MergeFrom( + *other_extension.lazymessage_value); + } else { + extension->message_value->CheckTypeAndMergeFrom( + other_extension.lazymessage_value->GetMessage( + *extension->message_value)); + } + } else { + if (extension->is_lazy) { + extension->lazymessage_value->MutableMessage( + *other_extension.message_value)->CheckTypeAndMergeFrom( + *other_extension.message_value); + } else { + extension->message_value->CheckTypeAndMergeFrom( + *other_extension.message_value); + } + } + } + extension->is_cleared = false; break; + } } } } @@ -706,7 +841,11 @@ bool ExtensionSet::IsInitialized() const { } } else { if (!extension.is_cleared) { - if (!extension.message_value->IsInitialized()) return false; + if (extension.is_lazy) { + if (!extension.lazymessage_value->IsInitialized()) return false; + } else { + if (!extension.message_value->IsInitialized()) return false; + } } } } @@ -715,27 +854,42 @@ bool ExtensionSet::IsInitialized() const { return true; } -bool ExtensionSet::ParseField(uint32 tag, io::CodedInputStream* input, - ExtensionFinder* extension_finder, - FieldSkipper* field_skipper) { - int number = WireFormatLite::GetTagFieldNumber(tag); +bool ExtensionSet::FindExtensionInfoFromTag( + uint32 tag, ExtensionFinder* extension_finder, + int* field_number, ExtensionInfo* extension) { + *field_number = WireFormatLite::GetTagFieldNumber(tag); WireFormatLite::WireType wire_type = WireFormatLite::GetTagWireType(tag); - ExtensionInfo extension; bool is_unknown; - if (!extension_finder->Find(number, &extension)) { + if (!extension_finder->Find(*field_number, extension)) { is_unknown = true; - } else if (extension.is_packed) { + } else if (extension->is_packed) { is_unknown = (wire_type != WireFormatLite::WIRETYPE_LENGTH_DELIMITED); } else { WireFormatLite::WireType expected_wire_type = - WireFormatLite::WireTypeForFieldType(real_type(extension.type)); + WireFormatLite::WireTypeForFieldType(real_type(extension->type)); is_unknown = (wire_type != expected_wire_type); } + return !is_unknown; +} + +bool ExtensionSet::ParseField(uint32 tag, io::CodedInputStream* input, + ExtensionFinder* extension_finder, + FieldSkipper* field_skipper) { + int number; + ExtensionInfo extension; + if (!FindExtensionInfoFromTag(tag, extension_finder, &number, &extension)) { + return field_skipper->SkipField(input, tag); + } else { + return ParseFieldWithExtensionInfo(number, extension, input, field_skipper); + } +} - if (is_unknown) { - field_skipper->SkipField(input, tag); - } else if (extension.is_packed) { +bool ExtensionSet::ParseFieldWithExtensionInfo( + int number, const ExtensionInfo& extension, + io::CodedInputStream* input, + FieldSkipper* field_skipper) { + if (extension.is_packed) { uint32 size; if (!input->ReadVarint32(&size)) return false; io::CodedInputStream::Limit limit = input->PushLimit(size); @@ -852,8 +1006,8 @@ bool ExtensionSet::ParseField(uint32 tag, io::CodedInputStream* input, case WireFormatLite::TYPE_BYTES: { string* value = extension.is_repeated ? - AddString(number, WireFormatLite::TYPE_STRING, extension.descriptor) : - MutableString(number, WireFormatLite::TYPE_STRING, + AddString(number, WireFormatLite::TYPE_BYTES, extension.descriptor) : + MutableString(number, WireFormatLite::TYPE_BYTES, extension.descriptor); if (!WireFormatLite::ReadBytes(input, value)) return false; break; @@ -897,121 +1051,11 @@ bool ExtensionSet::ParseField(uint32 tag, io::CodedInputStream* input, // const Message* containing_type, // UnknownFieldSet* unknown_fields) -bool ExtensionSet::ParseMessageSet(io::CodedInputStream* input, - ExtensionFinder* extension_finder, - FieldSkipper* field_skipper) { - while (true) { - uint32 tag = input->ReadTag(); - switch (tag) { - case 0: - return true; - case WireFormatLite::kMessageSetItemStartTag: - if (!ParseMessageSetItem(input, extension_finder, field_skipper)) { - return false; - } - break; - default: - if (!ParseField(tag, input, extension_finder, field_skipper)) { - return false; - } - break; - } - } -} - -bool ExtensionSet::ParseMessageSet(io::CodedInputStream* input, - const MessageLite* containing_type, - UnknownFieldSet* unknown_fields) { - FieldSkipper skipper(unknown_fields); - GeneratedExtensionFinder finder(containing_type); - return ParseMessageSet(input, &finder, &skipper); -} - // Defined in extension_set_heavy.cc. // bool ExtensionSet::ParseMessageSetHeavy(io::CodedInputStream* input, // const Message* containing_type, // UnknownFieldSet* unknown_fields); -bool ExtensionSet::ParseMessageSetItem(io::CodedInputStream* input, - ExtensionFinder* extension_finder, - FieldSkipper* field_skipper) { - // TODO(kenton): It would be nice to share code between this and - // WireFormatLite::ParseAndMergeMessageSetItem(), but I think the - // differences would be hard to factor out. - - // This method parses a group which should contain two fields: - // required int32 type_id = 2; - // required data message = 3; - - // Once we see a type_id, we'll construct a fake tag for this extension - // which is the tag it would have had under the proto2 extensions wire - // format. - uint32 fake_tag = 0; - - // If we see message data before the type_id, we'll append it to this so - // we can parse it later. This will probably never happen in practice, - // as no MessageSet encoder I know of writes the message before the type ID. - // But, it's technically valid so we should allow it. - // TODO(kenton): Use a Cord instead? Do I care? - string message_data; - - while (true) { - uint32 tag = input->ReadTag(); - if (tag == 0) return false; - - switch (tag) { - case WireFormatLite::kMessageSetTypeIdTag: { - uint32 type_id; - if (!input->ReadVarint32(&type_id)) return false; - fake_tag = WireFormatLite::MakeTag(type_id, - WireFormatLite::WIRETYPE_LENGTH_DELIMITED); - - if (!message_data.empty()) { - // We saw some message data before the type_id. Have to parse it - // now. - io::CodedInputStream sub_input( - reinterpret_cast<const uint8*>(message_data.data()), - message_data.size()); - if (!ParseField(fake_tag, &sub_input, - extension_finder, field_skipper)) { - return false; - } - message_data.clear(); - } - - break; - } - - case WireFormatLite::kMessageSetMessageTag: { - if (fake_tag == 0) { - // We haven't seen a type_id yet. Append this data to message_data. - string temp; - uint32 length; - if (!input->ReadVarint32(&length)) return false; - if (!input->ReadString(&temp, length)) return false; - message_data.append(temp); - } else { - // Already saw type_id, so we can parse this directly. - if (!ParseField(fake_tag, input, - extension_finder, field_skipper)) { - return false; - } - } - - break; - } - - case WireFormatLite::kMessageSetItemEndTag: { - return true; - } - - default: { - if (!field_skipper->SkipField(input, tag)) return false; - } - } - } -} - void ExtensionSet::SerializeWithCachedSizes( int start_field_number, int end_field_number, io::CodedOutputStream* output) const { @@ -1023,14 +1067,6 @@ void ExtensionSet::SerializeWithCachedSizes( } } -void ExtensionSet::SerializeMessageSetWithCachedSizes( - io::CodedOutputStream* output) const { - map<int, Extension>::const_iterator iter; - for (iter = extensions_.begin(); iter != extensions_.end(); ++iter) { - iter->second.SerializeMessageSetItemWithCachedSizes(iter->first, output); - } -} - int ExtensionSet::ByteSize() const { int total_size = 0; @@ -1042,17 +1078,6 @@ int ExtensionSet::ByteSize() const { return total_size; } -int ExtensionSet::MessageSetByteSize() const { - int total_size = 0; - - for (map<int, Extension>::const_iterator iter = extensions_.begin(); - iter != extensions_.end(); ++iter) { - total_size += iter->second.MessageSetItemByteSize(iter->first); - } - - return total_size; -} - // Defined in extension_set_heavy.cc. // int ExtensionSet::SpaceUsedExcludingSelf() const @@ -1096,7 +1121,11 @@ void ExtensionSet::Extension::Clear() { string_value->clear(); break; case WireFormatLite::CPPTYPE_MESSAGE: - message_value->Clear(); + if (is_lazy) { + lazymessage_value->Clear(); + } else { + message_value->Clear(); + } break; default: // No need to do anything. Get*() will return the default value @@ -1208,40 +1237,18 @@ void ExtensionSet::Extension::SerializeFieldWithCachedSizes( HANDLE_TYPE( BYTES, Bytes, *string_value); HANDLE_TYPE( ENUM, Enum, enum_value); HANDLE_TYPE( GROUP, Group, *message_value); - HANDLE_TYPE( MESSAGE, Message, *message_value); #undef HANDLE_TYPE + case WireFormatLite::TYPE_MESSAGE: + if (is_lazy) { + lazymessage_value->WriteMessage(number, output); + } else { + WireFormatLite::WriteMessage(number, *message_value, output); + } + break; } } } -void ExtensionSet::Extension::SerializeMessageSetItemWithCachedSizes( - int number, - io::CodedOutputStream* output) const { - if (type != WireFormatLite::TYPE_MESSAGE || is_repeated) { - // Not a valid MessageSet extension, but serialize it the normal way. - SerializeFieldWithCachedSizes(number, output); - return; - } - - if (is_cleared) return; - - // Start group. - output->WriteTag(WireFormatLite::kMessageSetItemStartTag); - - // Write type ID. - WireFormatLite::WriteUInt32(WireFormatLite::kMessageSetTypeIdNumber, - number, - output); - // Write message. - WireFormatLite::WriteMessageMaybeToArray( - WireFormatLite::kMessageSetMessageNumber, - *message_value, - output); - - // End group. - output->WriteTag(WireFormatLite::kMessageSetItemEndTag); -} - int ExtensionSet::Extension::ByteSize(int number) const { int result = 0; @@ -1355,8 +1362,16 @@ int ExtensionSet::Extension::ByteSize(int number) const { HANDLE_TYPE( BYTES, Bytes, *string_value); HANDLE_TYPE( ENUM, Enum, enum_value); HANDLE_TYPE( GROUP, Group, *message_value); - HANDLE_TYPE( MESSAGE, Message, *message_value); #undef HANDLE_TYPE + case WireFormatLite::TYPE_MESSAGE: { + if (is_lazy) { + int size = lazymessage_value->ByteSize(); + result += io::CodedOutputStream::VarintSize32(size) + size; + } else { + result += WireFormatLite::MessageSize(*message_value); + } + break; + } // Stuff with fixed size. #define HANDLE_TYPE(UPPERCASE, CAMELCASE) \ @@ -1377,29 +1392,6 @@ int ExtensionSet::Extension::ByteSize(int number) const { return result; } -int ExtensionSet::Extension::MessageSetItemByteSize(int number) const { - if (type != WireFormatLite::TYPE_MESSAGE || is_repeated) { - // Not a valid MessageSet extension, but compute the byte size for it the - // normal way. - return ByteSize(number); - } - - if (is_cleared) return 0; - - int our_size = WireFormatLite::kMessageSetItemTagsSize; - - // type_id - our_size += io::CodedOutputStream::VarintSize32(number); - - // message - int message_size = message_value->ByteSize(); - - our_size += io::CodedOutputStream::VarintSize32(message_size); - our_size += message_size; - - return our_size; -} - int ExtensionSet::Extension::GetSize() const { GOOGLE_DCHECK(is_repeated); switch (cpp_type(type)) { @@ -1450,7 +1442,11 @@ void ExtensionSet::Extension::Free() { delete string_value; break; case WireFormatLite::CPPTYPE_MESSAGE: - delete message_value; + if (is_lazy) { + delete lazymessage_value; + } else { + delete message_value; + } break; default: break; diff --git a/third_party/protobuf/src/google/protobuf/extension_set.h b/third_party/protobuf/src/google/protobuf/extension_set.h index a1b6f35..f3935fb 100644 --- a/third_party/protobuf/src/google/protobuf/extension_set.h +++ b/third_party/protobuf/src/google/protobuf/extension_set.h @@ -89,8 +89,8 @@ typedef bool EnumValidityFuncWithArg(const void* arg, int number); // Information about a registered extension. struct ExtensionInfo { inline ExtensionInfo() {} - inline ExtensionInfo(FieldType type, bool is_repeated, bool is_packed) - : type(type), is_repeated(is_repeated), is_packed(is_packed), + inline ExtensionInfo(FieldType type_param, bool isrepeated, bool ispacked) + : type(type_param), is_repeated(isrepeated), is_packed(ispacked), descriptor(NULL) {} FieldType type; @@ -214,6 +214,7 @@ class LIBPROTOBUF_EXPORT ExtensionSet { bool Has(int number) const; int ExtensionSize(int number) const; // Size of a repeated extension. + int NumExtensions() const; // The number of extensions FieldType ExtensionType(int number) const; void ClearExtension(int number); @@ -251,10 +252,21 @@ class LIBPROTOBUF_EXPORT ExtensionSet { const MessageLite& prototype, desc); MessageLite* MutableMessage(const FieldDescriptor* decsriptor, MessageFactory* factory); + // Adds the given message to the ExtensionSet, taking ownership of the + // message object. Existing message with the same number will be deleted. + // If "message" is NULL, this is equivalent to "ClearExtension(number)". + void SetAllocatedMessage(int number, FieldType type, + const FieldDescriptor* descriptor, + MessageLite* message); + MessageLite* ReleaseMessage(int number, const MessageLite& prototype); + MessageLite* ReleaseMessage(const FieldDescriptor* descriptor, + MessageFactory* factory); #undef desc // repeated fields ------------------------------------------------- + void* MutableRawRepeatedField(int number); + int32 GetRepeatedInt32 (int number, int index) const; int64 GetRepeatedInt64 (int number, int index) const; uint32 GetRepeatedUInt32(int number, int index) const; @@ -296,6 +308,7 @@ class LIBPROTOBUF_EXPORT ExtensionSet { #undef desc void RemoveLast(int number); + MessageLite* ReleaseLast(int number); void SwapElements(int number, int index1, int index2); // ----------------------------------------------------------------- @@ -312,18 +325,18 @@ class LIBPROTOBUF_EXPORT ExtensionSet { void Swap(ExtensionSet* other); bool IsInitialized() const; - // Parses a single extension from the input. The input should start out - // positioned immediately after the tag. |containing_type| is the default - // instance for the containing message; it is used only to look up the - // extension by number. See RegisterExtension(), above. Unlike the other - // methods of ExtensionSet, this only works for generated message types -- - // it looks up extensions registered using RegisterExtension(). + // Parses a single extension from the input. The input should start out + // positioned immediately after the tag. bool ParseField(uint32 tag, io::CodedInputStream* input, ExtensionFinder* extension_finder, FieldSkipper* field_skipper); // Specific versions for lite or full messages (constructs the appropriate - // FieldSkipper automatically). + // FieldSkipper automatically). |containing_type| is the default + // instance for the containing message; it is used only to look up the + // extension by number. See RegisterExtension(), above. Unlike the other + // methods of ExtensionSet, this only works for generated message types -- + // it looks up extensions registered using RegisterExtension(). bool ParseField(uint32 tag, io::CodedInputStream* input, const MessageLite* containing_type, UnknownFieldSet* unknown_fields); @@ -384,18 +397,49 @@ class LIBPROTOBUF_EXPORT ExtensionSet { private: + // Interface of a lazily parsed singular message extension. + class LIBPROTOBUF_EXPORT LazyMessageExtension { + public: + LazyMessageExtension() {} + virtual ~LazyMessageExtension() {} + + virtual LazyMessageExtension* New() const = 0; + virtual const MessageLite& GetMessage( + const MessageLite& prototype) const = 0; + virtual MessageLite* MutableMessage(const MessageLite& prototype) = 0; + virtual void SetAllocatedMessage(MessageLite *message) = 0; + virtual MessageLite* ReleaseMessage(const MessageLite& prototype) = 0; + + virtual bool IsInitialized() const = 0; + virtual int ByteSize() const = 0; + virtual int SpaceUsed() const = 0; + + virtual void MergeFrom(const LazyMessageExtension& other) = 0; + virtual void Clear() = 0; + + virtual bool ReadMessage(const MessageLite& prototype, + io::CodedInputStream* input) = 0; + virtual void WriteMessage(int number, + io::CodedOutputStream* output) const = 0; + virtual uint8* WriteMessageToArray(int number, uint8* target) const = 0; + private: + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(LazyMessageExtension); + }; struct Extension { + // The order of these fields packs Extension into 24 bytes when using 8 + // byte alignment. Consider this when adding or removing fields here. union { - int32 int32_value; - int64 int64_value; - uint32 uint32_value; - uint64 uint64_value; - float float_value; - double double_value; - bool bool_value; - int enum_value; - string* string_value; - MessageLite* message_value; + int32 int32_value; + int64 int64_value; + uint32 uint32_value; + uint64 uint64_value; + float float_value; + double double_value; + bool bool_value; + int enum_value; + string* string_value; + MessageLite* message_value; + LazyMessageExtension* lazymessage_value; RepeatedField <int32 >* repeated_int32_value; RepeatedField <int64 >* repeated_int64_value; @@ -418,21 +462,28 @@ class LIBPROTOBUF_EXPORT ExtensionSet { // removing it from the map, we just set is_cleared = true. This has no // meaning for repeated types; for those, the size of the RepeatedField // simply becomes zero when cleared. - bool is_cleared; + bool is_cleared : 4; + + // For singular message types, indicates whether lazy parsing is enabled + // for this extension. This field is only valid when type == TYPE_MESSAGE + // and !is_repeated because we only support lazy parsing for singular + // message types currently. If is_lazy = true, the extension is stored in + // lazymessage_value. Otherwise, the extension will be message_value. + bool is_lazy : 4; // For repeated types, this indicates if the [packed=true] option is set. bool is_packed; - // The descriptor for this extension, if one exists and is known. May be - // NULL. Must not be NULL if the descriptor for the extension does not - // live in the same pool as the descriptor for the containing type. - const FieldDescriptor* descriptor; - // For packed fields, the size of the packed data is recorded here when // ByteSize() is called then used during serialization. // TODO(kenton): Use atomic<int> when C++ supports it. mutable int cached_size; + // The descriptor for this extension, if one exists and is known. May be + // NULL. Must not be NULL if the descriptor for the extension does not + // live in the same pool as the descriptor for the containing type. + const FieldDescriptor* descriptor; + // Some helper methods for operations on a single Extension. void SerializeFieldWithCachedSizes( int number, @@ -455,6 +506,25 @@ class LIBPROTOBUF_EXPORT ExtensionSet { }; + // Returns true and fills field_number and extension if extension is found. + bool FindExtensionInfoFromTag(uint32 tag, ExtensionFinder* extension_finder, + int* field_number, ExtensionInfo* extension); + + // Parses a single extension from the input. The input should start out + // positioned immediately after the wire tag. This method is called in + // ParseField() after field number is extracted from the wire tag and + // ExtensionInfo is found by the field number. + bool ParseFieldWithExtensionInfo(int field_number, + const ExtensionInfo& extension, + io::CodedInputStream* input, + FieldSkipper* field_skipper); + + // Like ParseField(), but this method may parse singular message extensions + // lazily depending on the value of FLAGS_eagerly_parse_message_sets. + bool ParseFieldMaybeLazily(uint32 tag, io::CodedInputStream* input, + ExtensionFinder* extension_finder, + FieldSkipper* field_skipper); + // Gets the extension with the given number, creating it if it does not // already exist. Returns true if the extension did not already exist. bool MaybeNewExtension(int number, const FieldDescriptor* descriptor, @@ -722,6 +792,15 @@ class MessageTypeTraits { return static_cast<Type*>( set->MutableMessage(number, field_type, Type::default_instance(), NULL)); } + static inline void SetAllocated(int number, FieldType field_type, + MutableType message, ExtensionSet* set) { + set->SetAllocatedMessage(number, field_type, NULL, message); + } + static inline MutableType Release(int number, FieldType field_type, + ExtensionSet* set) { + return static_cast<Type*>(set->ReleaseMessage( + number, Type::default_instance())); + } }; template <typename Type> @@ -789,113 +868,137 @@ class ExtensionIdentifier { // causes problems if the class has a nested message or enum type with that // name and "_TypeTraits" is technically reserved for the C++ library since // it starts with an underscore followed by a capital letter. +// +// For similar reason, we use "_field_type" and "_is_packed" as parameter names +// below, so that "field_type" and "is_packed" can be used as field names. #define GOOGLE_PROTOBUF_EXTENSION_ACCESSORS(CLASSNAME) \ /* Has, Size, Clear */ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline bool HasExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) const { \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) const { \ return _extensions_.Has(id.number()); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline void ClearExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) { \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) { \ _extensions_.ClearExtension(id.number()); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline int ExtensionSize( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) const { \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) const { \ return _extensions_.ExtensionSize(id.number()); \ } \ \ /* Singular accessors */ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline typename _proto_TypeTraits::ConstType GetExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) const { \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) const { \ return _proto_TypeTraits::Get(id.number(), _extensions_, \ id.default_value()); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline typename _proto_TypeTraits::MutableType MutableExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) { \ - return _proto_TypeTraits::Mutable(id.number(), field_type, &_extensions_);\ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) { \ + return _proto_TypeTraits::Mutable(id.number(), _field_type, \ + &_extensions_); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline void SetExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id, \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ typename _proto_TypeTraits::ConstType value) { \ - _proto_TypeTraits::Set(id.number(), field_type, value, &_extensions_); \ + _proto_TypeTraits::Set(id.number(), _field_type, value, &_extensions_); \ + } \ + \ + template <typename _proto_TypeTraits, \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ + inline void SetAllocatedExtension( \ + const ::google::protobuf::internal::ExtensionIdentifier< \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ + typename _proto_TypeTraits::MutableType value) { \ + _proto_TypeTraits::SetAllocated(id.number(), _field_type, \ + value, &_extensions_); \ + } \ + template <typename _proto_TypeTraits, \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ + inline typename _proto_TypeTraits::MutableType ReleaseExtension( \ + const ::google::protobuf::internal::ExtensionIdentifier< \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) { \ + return _proto_TypeTraits::Release(id.number(), _field_type, \ + &_extensions_); \ } \ \ /* Repeated accessors */ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline typename _proto_TypeTraits::ConstType GetExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id, \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ int index) const { \ return _proto_TypeTraits::Get(id.number(), _extensions_, index); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline typename _proto_TypeTraits::MutableType MutableExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id, \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ int index) { \ return _proto_TypeTraits::Mutable(id.number(), index, &_extensions_); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline void SetExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id, \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ int index, typename _proto_TypeTraits::ConstType value) { \ _proto_TypeTraits::Set(id.number(), index, value, &_extensions_); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline typename _proto_TypeTraits::MutableType AddExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id) { \ - return _proto_TypeTraits::Add(id.number(), field_type, &_extensions_); \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id) { \ + return _proto_TypeTraits::Add(id.number(), _field_type, &_extensions_); \ } \ \ template <typename _proto_TypeTraits, \ - ::google::protobuf::internal::FieldType field_type, \ - bool is_packed> \ + ::google::protobuf::internal::FieldType _field_type, \ + bool _is_packed> \ inline void AddExtension( \ const ::google::protobuf::internal::ExtensionIdentifier< \ - CLASSNAME, _proto_TypeTraits, field_type, is_packed>& id, \ + CLASSNAME, _proto_TypeTraits, _field_type, _is_packed>& id, \ typename _proto_TypeTraits::ConstType value) { \ - _proto_TypeTraits::Add(id.number(), field_type, is_packed, \ + _proto_TypeTraits::Add(id.number(), _field_type, _is_packed, \ value, &_extensions_); \ } diff --git a/third_party/protobuf/src/google/protobuf/extension_set_heavy.cc b/third_party/protobuf/src/google/protobuf/extension_set_heavy.cc index e00fd4d..01b9490 100644 --- a/third_party/protobuf/src/google/protobuf/extension_set_heavy.cc +++ b/third_party/protobuf/src/google/protobuf/extension_set_heavy.cc @@ -35,17 +35,20 @@ // Contains methods defined in extension_set.h which cannot be part of the // lite library because they use descriptors or reflection. -#include <google/protobuf/extension_set.h> +#include <google/protobuf/io/zero_copy_stream_impl_lite.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/extension_set.h> #include <google/protobuf/message.h> #include <google/protobuf/repeated_field.h> #include <google/protobuf/wire_format.h> #include <google/protobuf/wire_format_lite_inl.h> namespace google { + namespace protobuf { namespace internal { + // Implementation of ExtensionFinder which finds extensions in a given // DescriptorPool, using the given MessageFactory to construct sub-objects. // This class is implemented in extension_set_heavy.cc. @@ -103,6 +106,11 @@ inline FieldDescriptor::CppType cpp_type(FieldType type) { static_cast<FieldDescriptor::Type>(type)); } +inline WireFormatLite::FieldType field_type(FieldType type) { + GOOGLE_DCHECK(type > 0 && type <= WireFormatLite::MAX_FIELD_TYPE); + return static_cast<WireFormatLite::FieldType>(type); +} + #define GOOGLE_DCHECK_TYPE(EXTENSION, LABEL, CPPTYPE) \ GOOGLE_DCHECK_EQ((EXTENSION).is_repeated ? FieldDescriptor::LABEL_REPEATED \ : FieldDescriptor::LABEL_OPTIONAL, \ @@ -118,7 +126,12 @@ const MessageLite& ExtensionSet::GetMessage(int number, return *factory->GetPrototype(message_type); } else { GOOGLE_DCHECK_TYPE(iter->second, OPTIONAL, MESSAGE); - return *iter->second.message_value; + if (iter->second.is_lazy) { + return iter->second.lazymessage_value->GetMessage( + *factory->GetPrototype(message_type)); + } else { + return *iter->second.message_value; + } } } @@ -132,13 +145,41 @@ MessageLite* ExtensionSet::MutableMessage(const FieldDescriptor* descriptor, extension->is_packed = false; const MessageLite* prototype = factory->GetPrototype(descriptor->message_type()); - GOOGLE_CHECK(prototype != NULL); + extension->is_lazy = false; extension->message_value = prototype->New(); + extension->is_cleared = false; + return extension->message_value; } else { GOOGLE_DCHECK_TYPE(*extension, OPTIONAL, MESSAGE); + extension->is_cleared = false; + if (extension->is_lazy) { + return extension->lazymessage_value->MutableMessage( + *factory->GetPrototype(descriptor->message_type())); + } else { + return extension->message_value; + } + } +} + +MessageLite* ExtensionSet::ReleaseMessage(const FieldDescriptor* descriptor, + MessageFactory* factory) { + map<int, Extension>::iterator iter = extensions_.find(descriptor->number()); + if (iter == extensions_.end()) { + // Not present. Return NULL. + return NULL; + } else { + GOOGLE_DCHECK_TYPE(iter->second, OPTIONAL, MESSAGE); + MessageLite* ret = NULL; + if (iter->second.is_lazy) { + ret = iter->second.lazymessage_value->ReleaseMessage( + *factory->GetPrototype(descriptor->message_type())); + delete iter->second.lazymessage_value; + } else { + ret = iter->second.message_value; + } + extensions_.erase(descriptor->number()); + return ret; } - extension->is_cleared = false; - return extension->message_value; } MessageLite* ExtensionSet::AddMessage(const FieldDescriptor* descriptor, @@ -157,7 +198,7 @@ MessageLite* ExtensionSet::AddMessage(const FieldDescriptor* descriptor, // RepeatedPtrField<Message> does not know how to Add() since it cannot // allocate an abstract object, so we have to be tricky. MessageLite* result = extension->repeated_message_value - ->AddFromCleared<internal::GenericTypeHandler<MessageLite> >(); + ->AddFromCleared<GenericTypeHandler<MessageLite> >(); if (result == NULL) { const MessageLite* prototype; if (extension->repeated_message_value->size() == 0) { @@ -286,7 +327,11 @@ int ExtensionSet::Extension::SpaceUsedExcludingSelf() const { StringSpaceUsedExcludingSelf(*string_value); break; case FieldDescriptor::CPPTYPE_MESSAGE: - total_size += down_cast<Message*>(message_value)->SpaceUsed(); + if (is_lazy) { + total_size += lazymessage_value->SpaceUsed(); + } else { + total_size += down_cast<Message*>(message_value)->SpaceUsed(); + } break; default: // No extra storage costs for primitive types. @@ -419,8 +464,15 @@ uint8* ExtensionSet::Extension::SerializeFieldWithCachedSizesToArray( HANDLE_TYPE( BYTES, Bytes, *string_value); HANDLE_TYPE( ENUM, Enum, enum_value); HANDLE_TYPE( GROUP, Group, *message_value); - HANDLE_TYPE( MESSAGE, Message, *message_value); #undef HANDLE_TYPE + case FieldDescriptor::TYPE_MESSAGE: + if (is_lazy) { + target = lazymessage_value->WriteMessageToArray(number, target); + } else { + target = WireFormatLite::WriteMessageToArray( + number, *message_value, target); + } + break; } } return target; @@ -444,14 +496,217 @@ uint8* ExtensionSet::Extension::SerializeMessageSetItemWithCachedSizesToArray( target = WireFormatLite::WriteUInt32ToArray( WireFormatLite::kMessageSetTypeIdNumber, number, target); // Write message. - target = WireFormatLite::WriteMessageToArray( - WireFormatLite::kMessageSetMessageNumber, *message_value, target); + if (is_lazy) { + target = lazymessage_value->WriteMessageToArray( + WireFormatLite::kMessageSetMessageNumber, target); + } else { + target = WireFormatLite::WriteMessageToArray( + WireFormatLite::kMessageSetMessageNumber, *message_value, target); + } // End group. target = io::CodedOutputStream::WriteTagToArray( WireFormatLite::kMessageSetItemEndTag, target); return target; } + +bool ExtensionSet::ParseFieldMaybeLazily( + uint32 tag, io::CodedInputStream* input, + ExtensionFinder* extension_finder, + FieldSkipper* field_skipper) { + return ParseField(tag, input, extension_finder, field_skipper); +} + +bool ExtensionSet::ParseMessageSet(io::CodedInputStream* input, + ExtensionFinder* extension_finder, + FieldSkipper* field_skipper) { + while (true) { + uint32 tag = input->ReadTag(); + switch (tag) { + case 0: + return true; + case WireFormatLite::kMessageSetItemStartTag: + if (!ParseMessageSetItem(input, extension_finder, field_skipper)) { + return false; + } + break; + default: + if (!ParseField(tag, input, extension_finder, field_skipper)) { + return false; + } + break; + } + } +} + +bool ExtensionSet::ParseMessageSet(io::CodedInputStream* input, + const MessageLite* containing_type, + UnknownFieldSet* unknown_fields) { + FieldSkipper skipper(unknown_fields); + GeneratedExtensionFinder finder(containing_type); + return ParseMessageSet(input, &finder, &skipper); +} + +bool ExtensionSet::ParseMessageSetItem(io::CodedInputStream* input, + ExtensionFinder* extension_finder, + FieldSkipper* field_skipper) { + // TODO(kenton): It would be nice to share code between this and + // WireFormatLite::ParseAndMergeMessageSetItem(), but I think the + // differences would be hard to factor out. + + // This method parses a group which should contain two fields: + // required int32 type_id = 2; + // required data message = 3; + + // Once we see a type_id, we'll construct a fake tag for this extension + // which is the tag it would have had under the proto2 extensions wire + // format. + uint32 fake_tag = 0; + + // If we see message data before the type_id, we'll append it to this so + // we can parse it later. + string message_data; + + while (true) { + uint32 tag = input->ReadTag(); + if (tag == 0) return false; + + switch (tag) { + case WireFormatLite::kMessageSetTypeIdTag: { + uint32 type_id; + if (!input->ReadVarint32(&type_id)) return false; + fake_tag = WireFormatLite::MakeTag(type_id, + WireFormatLite::WIRETYPE_LENGTH_DELIMITED); + + if (!message_data.empty()) { + // We saw some message data before the type_id. Have to parse it + // now. + io::CodedInputStream sub_input( + reinterpret_cast<const uint8*>(message_data.data()), + message_data.size()); + if (!ParseFieldMaybeLazily(fake_tag, &sub_input, + extension_finder, field_skipper)) { + return false; + } + message_data.clear(); + } + + break; + } + + case WireFormatLite::kMessageSetMessageTag: { + if (fake_tag == 0) { + // We haven't seen a type_id yet. Append this data to message_data. + string temp; + uint32 length; + if (!input->ReadVarint32(&length)) return false; + if (!input->ReadString(&temp, length)) return false; + io::StringOutputStream output_stream(&message_data); + io::CodedOutputStream coded_output(&output_stream); + coded_output.WriteVarint32(length); + coded_output.WriteString(temp); + } else { + // Already saw type_id, so we can parse this directly. + if (!ParseFieldMaybeLazily(fake_tag, input, + extension_finder, field_skipper)) { + return false; + } + } + + break; + } + + case WireFormatLite::kMessageSetItemEndTag: { + return true; + } + + default: { + if (!field_skipper->SkipField(input, tag)) return false; + } + } + } +} + +void ExtensionSet::Extension::SerializeMessageSetItemWithCachedSizes( + int number, + io::CodedOutputStream* output) const { + if (type != WireFormatLite::TYPE_MESSAGE || is_repeated) { + // Not a valid MessageSet extension, but serialize it the normal way. + SerializeFieldWithCachedSizes(number, output); + return; + } + + if (is_cleared) return; + + // Start group. + output->WriteTag(WireFormatLite::kMessageSetItemStartTag); + + // Write type ID. + WireFormatLite::WriteUInt32(WireFormatLite::kMessageSetTypeIdNumber, + number, + output); + // Write message. + if (is_lazy) { + lazymessage_value->WriteMessage( + WireFormatLite::kMessageSetMessageNumber, output); + } else { + WireFormatLite::WriteMessageMaybeToArray( + WireFormatLite::kMessageSetMessageNumber, + *message_value, + output); + } + + // End group. + output->WriteTag(WireFormatLite::kMessageSetItemEndTag); +} + +int ExtensionSet::Extension::MessageSetItemByteSize(int number) const { + if (type != WireFormatLite::TYPE_MESSAGE || is_repeated) { + // Not a valid MessageSet extension, but compute the byte size for it the + // normal way. + return ByteSize(number); + } + + if (is_cleared) return 0; + + int our_size = WireFormatLite::kMessageSetItemTagsSize; + + // type_id + our_size += io::CodedOutputStream::VarintSize32(number); + + // message + int message_size = 0; + if (is_lazy) { + message_size = lazymessage_value->ByteSize(); + } else { + message_size = message_value->ByteSize(); + } + + our_size += io::CodedOutputStream::VarintSize32(message_size); + our_size += message_size; + + return our_size; +} + +void ExtensionSet::SerializeMessageSetWithCachedSizes( + io::CodedOutputStream* output) const { + map<int, Extension>::const_iterator iter; + for (iter = extensions_.begin(); iter != extensions_.end(); ++iter) { + iter->second.SerializeMessageSetItemWithCachedSizes(iter->first, output); + } +} + +int ExtensionSet::MessageSetByteSize() const { + int total_size = 0; + + for (map<int, Extension>::const_iterator iter = extensions_.begin(); + iter != extensions_.end(); ++iter) { + total_size += iter->second.MessageSetItemByteSize(iter->first); + } + + return total_size; +} + } // namespace internal } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/extension_set_unittest.cc b/third_party/protobuf/src/google/protobuf/extension_set_unittest.cc index 000f846..1e7c5a5 100644 --- a/third_party/protobuf/src/google/protobuf/extension_set_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/extension_set_unittest.cc @@ -34,6 +34,7 @@ #include <google/protobuf/extension_set.h> #include <google/protobuf/unittest.pb.h> +#include <google/protobuf/unittest_mset.pb.h> #include <google/protobuf/test_util.h> #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/descriptor.h> @@ -46,9 +47,10 @@ #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { + namespace protobuf { namespace internal { namespace { @@ -140,23 +142,98 @@ TEST(ExtensionSetTest, ClearOneField) { TestUtil::ExpectAllExtensionsSet(message); } +TEST(ExtensionSetTest, SetAllocatedExtensin) { + unittest::TestAllExtensions message; + EXPECT_FALSE(message.HasExtension( + unittest::optional_foreign_message_extension)); + // Add a extension using SetAllocatedExtension + unittest::ForeignMessage* foreign_message = new unittest::ForeignMessage(); + message.SetAllocatedExtension(unittest::optional_foreign_message_extension, + foreign_message); + EXPECT_TRUE(message.HasExtension( + unittest::optional_foreign_message_extension)); + EXPECT_EQ(foreign_message, + message.MutableExtension( + unittest::optional_foreign_message_extension)); + EXPECT_EQ(foreign_message, + &message.GetExtension( + unittest::optional_foreign_message_extension)); + + // SetAllocatedExtension should delete the previously existing extension. + // (We reply on unittest to check memory leaks for this case) + message.SetAllocatedExtension(unittest::optional_foreign_message_extension, + new unittest::ForeignMessage()); + + // SetAllocatedExtension with a NULL parameter is equivalent to ClearExtenion. + message.SetAllocatedExtension(unittest::optional_foreign_message_extension, + NULL); + EXPECT_FALSE(message.HasExtension( + unittest::optional_foreign_message_extension)); +} + +TEST(ExtensionSetTest, ReleaseExtension) { + unittest::TestMessageSet message; + EXPECT_FALSE(message.HasExtension( + unittest::TestMessageSetExtension1::message_set_extension)); + // Add a extension using SetAllocatedExtension + unittest::TestMessageSetExtension1* extension = + new unittest::TestMessageSetExtension1(); + message.SetAllocatedExtension( + unittest::TestMessageSetExtension1::message_set_extension, + extension); + EXPECT_TRUE(message.HasExtension( + unittest::TestMessageSetExtension1::message_set_extension)); + // Release the extension using ReleaseExtension + unittest::TestMessageSetExtension1* released_extension = + message.ReleaseExtension( + unittest::TestMessageSetExtension1::message_set_extension); + EXPECT_EQ(extension, released_extension); + EXPECT_FALSE(message.HasExtension( + unittest::TestMessageSetExtension1::message_set_extension)); + // ReleaseExtension will return the underlying object even after + // ClearExtension is called. + message.SetAllocatedExtension( + unittest::TestMessageSetExtension1::message_set_extension, + extension); + message.ClearExtension( + unittest::TestMessageSetExtension1::message_set_extension); + released_extension = message.ReleaseExtension( + unittest::TestMessageSetExtension1::message_set_extension); + EXPECT_TRUE(released_extension != NULL); + delete released_extension; +} + + TEST(ExtensionSetTest, CopyFrom) { unittest::TestAllExtensions message1, message2; - string data; TestUtil::SetAllExtensions(&message1); message2.CopyFrom(message1); TestUtil::ExpectAllExtensionsSet(message2); + message2.CopyFrom(message1); // exercise copy when fields already exist + TestUtil::ExpectAllExtensionsSet(message2); +} + +TEST(ExtensioSetTest, CopyFromPacked) { + unittest::TestPackedExtensions message1, message2; + + TestUtil::SetPackedExtensions(&message1); + message2.CopyFrom(message1); + TestUtil::ExpectPackedExtensionsSet(message2); + message2.CopyFrom(message1); // exercise copy when fields already exist + TestUtil::ExpectPackedExtensionsSet(message2); } TEST(ExtensionSetTest, CopyFromUpcasted) { unittest::TestAllExtensions message1, message2; - string data; const Message& upcasted_message = message1; TestUtil::SetAllExtensions(&message1); message2.CopyFrom(upcasted_message); TestUtil::ExpectAllExtensionsSet(message2); + // exercise copy when fields already exist + message2.CopyFrom(upcasted_message); + TestUtil::ExpectAllExtensionsSet(message2); } TEST(ExtensionSetTest, SwapWithEmpty) { @@ -418,7 +495,8 @@ TEST(ExtensionSetTest, SpaceUsedExcludingSelf) { for (int i = 0; i < 16; ++i) { \ message.AddExtension(unittest::repeated_##type##_extension, value); \ } \ - int expected_size = sizeof(cpptype) * 16 + empty_repeated_field_size; \ + int expected_size = sizeof(cpptype) * (16 - \ + kMinRepeatedFieldAllocationSize) + empty_repeated_field_size; \ EXPECT_EQ(expected_size, message.SpaceUsed()) << #type; \ } while (0) @@ -450,7 +528,8 @@ TEST(ExtensionSetTest, SpaceUsedExcludingSelf) { for (int i = 0; i < 16; ++i) { message.AddExtension(unittest::repeated_string_extension, value); } - min_expected_size += (sizeof(value) + value.size()) * 16; + min_expected_size += (sizeof(value) + value.size()) * + (16 - kMinRepeatedFieldAllocationSize); EXPECT_LE(min_expected_size, message.SpaceUsed()); } // Repeated messages @@ -465,7 +544,8 @@ TEST(ExtensionSetTest, SpaceUsedExcludingSelf) { message.AddExtension(unittest::repeated_foreign_message_extension)-> CopyFrom(prototype); } - min_expected_size += 16 * prototype.SpaceUsed(); + min_expected_size += + (16 - kMinRepeatedFieldAllocationSize) * prototype.SpaceUsed(); EXPECT_LE(min_expected_size, message.SpaceUsed()); } } diff --git a/third_party/protobuf/src/google/protobuf/generated_enum_reflection.h b/third_party/protobuf/src/google/protobuf/generated_enum_reflection.h new file mode 100644 index 0000000..a09a540 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/generated_enum_reflection.h @@ -0,0 +1,85 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: jasonh@google.com (Jason Hsueh) +// +// This header is logically internal, but is made public because it is used +// from protocol-compiler-generated code, which may reside in other components. +// It provides reflection support for generated enums, and is included in +// generated .pb.h files and should have minimal dependencies. The methods are +// implemented in generated_message_reflection.cc. + +#ifndef GOOGLE_PROTOBUF_GENERATED_ENUM_REFLECTION_H__ +#define GOOGLE_PROTOBUF_GENERATED_ENUM_REFLECTION_H__ + +#include <string> + +namespace google { +namespace protobuf { + class EnumDescriptor; +} // namespace protobuf + +namespace protobuf { + +// Returns the EnumDescriptor for enum type E, which must be a +// proto-declared enum type. Code generated by the protocol compiler +// will include specializations of this template for each enum type declared. +template <typename E> +const EnumDescriptor* GetEnumDescriptor(); + +namespace internal { + +// Helper for EnumType_Parse functions: try to parse the string 'name' as an +// enum name of the given type, returning true and filling in value on success, +// or returning false and leaving value unchanged on failure. +LIBPROTOBUF_EXPORT bool ParseNamedEnum(const EnumDescriptor* descriptor, + const string& name, + int* value); + +template<typename EnumType> +bool ParseNamedEnum(const EnumDescriptor* descriptor, + const string& name, + EnumType* value) { + int tmp; + if (!ParseNamedEnum(descriptor, name, &tmp)) return false; + *value = static_cast<EnumType>(tmp); + return true; +} + +// Just a wrapper around printing the name of a value. The main point of this +// function is not to be inlined, so that you can do this without including +// descriptor.h. +LIBPROTOBUF_EXPORT const string& NameOfEnum(const EnumDescriptor* descriptor, int value); + +} // namespace internal +} // namespace protobuf + +} // namespace google +#endif // GOOGLE_PROTOBUF_GENERATED_ENUM_REFLECTION_H__ diff --git a/third_party/protobuf/src/google/protobuf/generated_message_reflection.cc b/third_party/protobuf/src/google/protobuf/generated_message_reflection.cc index 226e951..09b0699 100644 --- a/third_party/protobuf/src/google/protobuf/generated_message_reflection.cc +++ b/third_party/protobuf/src/google/protobuf/generated_message_reflection.cc @@ -33,9 +33,9 @@ // Sanjay Ghemawat, Jeff Dean, and others. #include <algorithm> +#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/generated_message_reflection.h> #include <google/protobuf/descriptor.h> -#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/repeated_field.h> #include <google/protobuf/extension_set.h> #include <google/protobuf/generated_message_util.h> @@ -309,7 +309,7 @@ void GeneratedMessageReflection::Swap( "descriptor."; GOOGLE_CHECK_EQ(message2->GetReflection(), this) << "Second argument to Swap() (of type \"" - << message1->GetDescriptor()->full_name() + << message2->GetDescriptor()->full_name() << "\") is not compatible with this reflection object (which is for type \"" << descriptor_->full_name() << "\"). Note that the exact same class is required; not just the same " @@ -368,8 +368,11 @@ void GeneratedMessageReflection::Swap( SWAP_VALUES(DOUBLE, double); SWAP_VALUES(BOOL , bool ); SWAP_VALUES(ENUM , int ); - SWAP_VALUES(MESSAGE, Message*); #undef SWAP_VALUES + case FieldDescriptor::CPPTYPE_MESSAGE: + std::swap(*MutableRaw<Message*>(message1, field), + *MutableRaw<Message*>(message2, field)); + break; case FieldDescriptor::CPPTYPE_STRING: switch (field->options().ctype()) { @@ -576,6 +579,20 @@ void GeneratedMessageReflection::RemoveLast( } } +Message* GeneratedMessageReflection::ReleaseLast( + Message* message, + const FieldDescriptor* field) const { + USAGE_CHECK_ALL(ReleaseLast, REPEATED, MESSAGE); + + if (field->is_extension()) { + return static_cast<Message*>( + MutableExtensionSet(message)->ReleaseLast(field->number())); + } else { + return MutableRaw<RepeatedPtrFieldBase>(message, field) + ->ReleaseLast<GenericTypeHandler<Message> >(); + } +} + void GeneratedMessageReflection::SwapElements( Message* message, const FieldDescriptor* field, @@ -877,7 +894,9 @@ const EnumValueDescriptor* GeneratedMessageReflection::GetEnum( } const EnumValueDescriptor* result = field->enum_type()->FindValueByNumber(value); - GOOGLE_CHECK(result != NULL); + GOOGLE_CHECK(result != NULL) << "Value " << value << " is not valid for field " + << field->full_name() << " of type " + << field->enum_type()->full_name() << "."; return result; } @@ -907,7 +926,9 @@ const EnumValueDescriptor* GeneratedMessageReflection::GetRepeatedEnum( } const EnumValueDescriptor* result = field->enum_type()->FindValueByNumber(value); - GOOGLE_CHECK(result != NULL); + GOOGLE_CHECK(result != NULL) << "Value " << value << " is not valid for field " + << field->full_name() << " of type " + << field->enum_type()->full_name() << "."; return result; } @@ -948,13 +969,15 @@ const Message& GeneratedMessageReflection::GetMessage( MessageFactory* factory) const { USAGE_CHECK_ALL(GetMessage, SINGULAR, MESSAGE); + if (factory == NULL) factory = message_factory_; + if (field->is_extension()) { return static_cast<const Message&>( GetExtensionSet(message).GetMessage( - field->number(), field->message_type(), - factory == NULL ? message_factory_ : factory)); + field->number(), field->message_type(), factory)); } else { - const Message* result = GetRaw<const Message*>(message, field); + const Message* result; + result = GetRaw<const Message*>(message, field); if (result == NULL) { result = DefaultRaw<const Message*>(field); } @@ -967,17 +990,40 @@ Message* GeneratedMessageReflection::MutableMessage( MessageFactory* factory) const { USAGE_CHECK_ALL(MutableMessage, SINGULAR, MESSAGE); + if (factory == NULL) factory = message_factory_; + if (field->is_extension()) { return static_cast<Message*>( - MutableExtensionSet(message)->MutableMessage(field, - factory == NULL ? message_factory_ : factory)); + MutableExtensionSet(message)->MutableMessage(field, factory)); } else { - Message** result = MutableField<Message*>(message, field); - if (*result == NULL) { + Message* result; + Message** result_holder = MutableField<Message*>(message, field); + if (*result_holder == NULL) { const Message* default_message = DefaultRaw<const Message*>(field); - *result = default_message->New(); + *result_holder = default_message->New(); } - return *result; + result = *result_holder; + return result; + } +} + +Message* GeneratedMessageReflection::ReleaseMessage( + Message* message, + const FieldDescriptor* field, + MessageFactory* factory) const { + USAGE_CHECK_ALL(ReleaseMessage, SINGULAR, MESSAGE); + + if (factory == NULL) factory = message_factory_; + + if (field->is_extension()) { + return static_cast<Message*>( + MutableExtensionSet(message)->ReleaseMessage(field, factory)); + } else { + ClearBit(message, field); + Message** result = MutableRaw<Message*>(message, field); + Message* ret = *result; + *result = NULL; + return ret; } } @@ -1022,7 +1068,7 @@ Message* GeneratedMessageReflection::AddMessage( // We can't use AddField<Message>() because RepeatedPtrFieldBase doesn't // know how to allocate one. RepeatedPtrFieldBase* repeated = - MutableRaw<RepeatedPtrFieldBase>(message, field); + MutableRaw<RepeatedPtrFieldBase>(message, field); Message* result = repeated->AddFromCleared<GenericTypeHandler<Message> >(); if (result == NULL) { // We must allocate a new object. @@ -1039,7 +1085,26 @@ Message* GeneratedMessageReflection::AddMessage( } } -// ------------------------------------------------------------------- +void* GeneratedMessageReflection::MutableRawRepeatedField( + Message* message, const FieldDescriptor* field, + FieldDescriptor::CppType cpptype, + int ctype, const Descriptor* desc) const { + USAGE_CHECK_REPEATED("MutableRawRepeatedField"); + if (field->cpp_type() != cpptype) + ReportReflectionUsageTypeError(descriptor_, + field, "MutableRawRepeatedField", cpptype); + if (ctype >= 0) + GOOGLE_CHECK_EQ(field->options().ctype(), ctype) << "subtype mismatch"; + if (desc != NULL) + GOOGLE_CHECK_EQ(field->message_type(), desc) << "wrong submessage type"; + if (field->is_extension()) + return MutableExtensionSet(message)->MutableRawRepeatedField( + field->number()); + else + return reinterpret_cast<uint8*>(message) + offsets_[field->index()]; +} + +// ----------------------------------------------------------------------------- const FieldDescriptor* GeneratedMessageReflection::FindKnownExtensionByName( const string& name) const { diff --git a/third_party/protobuf/src/google/protobuf/generated_message_reflection.h b/third_party/protobuf/src/google/protobuf/generated_message_reflection.h index b545fa1..e6670ae 100644 --- a/third_party/protobuf/src/google/protobuf/generated_message_reflection.h +++ b/third_party/protobuf/src/google/protobuf/generated_message_reflection.h @@ -40,17 +40,23 @@ #include <string> #include <vector> +#include <google/protobuf/stubs/common.h> +// TODO(jasonh): Remove this once the compiler change to directly include this +// is released to components. +#include <google/protobuf/generated_enum_reflection.h> #include <google/protobuf/message.h> #include <google/protobuf/unknown_field_set.h> namespace google { +namespace upb { +namespace proto2_bridge_opensource { +class FieldAccessor; +} // namespace protobuf_bridge_google3 +} // namespace upb + namespace protobuf { class DescriptorPool; - // Generated code needs these to have been forward-declared. Easier to do it - // here than to print them inside every .pb.h file. - class FileDescriptor; - class EnumDescriptor; } namespace protobuf { @@ -141,6 +147,7 @@ class LIBPROTOBUF_EXPORT GeneratedMessageReflection : public Reflection { int FieldSize(const Message& message, const FieldDescriptor* field) const; void ClearField(Message* message, const FieldDescriptor* field) const; void RemoveLast(Message* message, const FieldDescriptor* field) const; + Message* ReleaseLast(Message* message, const FieldDescriptor* field) const; void Swap(Message* message1, Message* message2) const; void SwapElements(Message* message, const FieldDescriptor* field, int index1, int index2) const; @@ -193,6 +200,8 @@ class LIBPROTOBUF_EXPORT GeneratedMessageReflection : public Reflection { const EnumValueDescriptor* value) const; Message* MutableMessage(Message* message, const FieldDescriptor* field, MessageFactory* factory = NULL) const; + Message* ReleaseMessage(Message* message, const FieldDescriptor* field, + MessageFactory* factory = NULL) const; int32 GetRepeatedInt32 (const Message& message, const FieldDescriptor* field, int index) const; @@ -270,9 +279,18 @@ class LIBPROTOBUF_EXPORT GeneratedMessageReflection : public Reflection { const FieldDescriptor* FindKnownExtensionByName(const string& name) const; const FieldDescriptor* FindKnownExtensionByNumber(int number) const; + protected: + virtual void* MutableRawRepeatedField( + Message* message, const FieldDescriptor* field, FieldDescriptor::CppType, + int ctype, const Descriptor* desc) const; + private: friend class GeneratedMessage; + // To parse directly into a proto2 generated class, FieldAccessor needs + // access to member offsets and hasbits. + friend class LIBPROTOBUF_EXPORT upb::proto2_bridge_opensource::FieldAccessor; + const Descriptor* descriptor_; const Message* default_instance_; const int* offsets_; @@ -293,7 +311,6 @@ class LIBPROTOBUF_EXPORT GeneratedMessageReflection : public Reflection { const FieldDescriptor* field) const; template <typename Type> inline const Type& DefaultRaw(const FieldDescriptor* field) const; - inline const Message* GetMessagePrototype(const FieldDescriptor* field) const; inline const uint32* GetHasBits(const Message& message) const; inline uint32* MutableHasBits(Message* message) const; @@ -395,28 +412,6 @@ inline To dynamic_cast_if_available(From from) { #endif } -// Helper for EnumType_Parse functions: try to parse the string 'name' as an -// enum name of the given type, returning true and filling in value on success, -// or returning false and leaving value unchanged on failure. -LIBPROTOBUF_EXPORT bool ParseNamedEnum(const EnumDescriptor* descriptor, - const string& name, - int* value); - -template<typename EnumType> -bool ParseNamedEnum(const EnumDescriptor* descriptor, - const string& name, - EnumType* value) { - int tmp; - if (!ParseNamedEnum(descriptor, name, &tmp)) return false; - *value = static_cast<EnumType>(tmp); - return true; -} - -// Just a wrapper around printing the name of a value. The main point of this -// function is not to be inlined, so that you can do this without including -// descriptor.h. -LIBPROTOBUF_EXPORT const string& NameOfEnum(const EnumDescriptor* descriptor, int value); - } // namespace internal } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/generated_message_reflection_unittest.cc b/third_party/protobuf/src/google/protobuf/generated_message_reflection_unittest.cc index a03bcdb..f60e430 100644 --- a/third_party/protobuf/src/google/protobuf/generated_message_reflection_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/generated_message_reflection_unittest.cc @@ -225,11 +225,59 @@ TEST(GeneratedMessageReflectionTest, RemoveLastExtensions) { unittest::TestAllExtensions::descriptor()); TestUtil::SetAllExtensions(&message); + reflection_tester.RemoveLastRepeatedsViaReflection(&message); TestUtil::ExpectLastRepeatedExtensionsRemoved(message); } +TEST(GeneratedMessageReflectionTest, ReleaseLast) { + unittest::TestAllTypes message; + const Descriptor* descriptor = message.GetDescriptor(); + TestUtil::ReflectionTester reflection_tester(descriptor); + + TestUtil::SetAllFields(&message); + + reflection_tester.ReleaseLastRepeatedsViaReflection(&message, false); + + TestUtil::ExpectLastRepeatedsReleased(message); + + // Now test that we actually release the right message. + message.Clear(); + TestUtil::SetAllFields(&message); + ASSERT_EQ(2, message.repeated_foreign_message_size()); + const protobuf_unittest::ForeignMessage* expected = + message.mutable_repeated_foreign_message(1); + scoped_ptr<Message> released(message.GetReflection()->ReleaseLast( + &message, descriptor->FindFieldByName("repeated_foreign_message"))); + EXPECT_EQ(expected, released.get()); +} + +TEST(GeneratedMessageReflectionTest, ReleaseLastExtensions) { + unittest::TestAllExtensions message; + const Descriptor* descriptor = message.GetDescriptor(); + TestUtil::ReflectionTester reflection_tester(descriptor); + + TestUtil::SetAllExtensions(&message); + + reflection_tester.ReleaseLastRepeatedsViaReflection(&message, true); + + TestUtil::ExpectLastRepeatedExtensionsReleased(message); + + // Now test that we actually release the right message. + message.Clear(); + TestUtil::SetAllExtensions(&message); + ASSERT_EQ(2, message.ExtensionSize( + unittest::repeated_foreign_message_extension)); + const protobuf_unittest::ForeignMessage* expected = message.MutableExtension( + unittest::repeated_foreign_message_extension, 1); + scoped_ptr<Message> released(message.GetReflection()->ReleaseLast( + &message, descriptor->file()->FindExtensionByName( + "repeated_foreign_message_extension"))); + EXPECT_EQ(expected, released.get()); + +} + TEST(GeneratedMessageReflectionTest, SwapRepeatedElements) { unittest::TestAllTypes message; TestUtil::ReflectionTester reflection_tester( @@ -327,6 +375,58 @@ TEST(GeneratedMessageReflectionTest, FindKnownExtensionByName) { FindKnownExtensionByName(extension1->full_name()) == NULL); } +TEST(GeneratedMessageReflectionTest, ReleaseMessageTest) { + unittest::TestAllTypes message; + TestUtil::ReflectionTester reflection_tester( + unittest::TestAllTypes::descriptor()); + + // When nothing is set, we expect all released messages to be NULL. + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::IS_NULL); + + // After fields are set we should get non-NULL releases. + reflection_tester.SetAllFieldsViaReflection(&message); + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::NOT_NULL); + + // After Clear() we may or may not get a message from ReleaseMessage(). + // This is implementation specific. + reflection_tester.SetAllFieldsViaReflection(&message); + message.Clear(); + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::CAN_BE_NULL); + + // Test a different code path for setting after releasing. + TestUtil::SetAllFields(&message); + TestUtil::ExpectAllFieldsSet(message); +} + +TEST(GeneratedMessageReflectionTest, ReleaseExtensionMessageTest) { + unittest::TestAllExtensions message; + TestUtil::ReflectionTester reflection_tester( + unittest::TestAllExtensions::descriptor()); + + // When nothing is set, we expect all released messages to be NULL. + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::IS_NULL); + + // After fields are set we should get non-NULL releases. + reflection_tester.SetAllFieldsViaReflection(&message); + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::NOT_NULL); + + // After Clear() we may or may not get a message from ReleaseMessage(). + // This is implementation specific. + reflection_tester.SetAllFieldsViaReflection(&message); + message.Clear(); + reflection_tester.ExpectMessagesReleasedViaReflection( + &message, TestUtil::ReflectionTester::CAN_BE_NULL); + + // Test a different code path for setting after releasing. + TestUtil::SetAllExtensions(&message); + TestUtil::ExpectAllExtensionsSet(message); +} + #ifdef GTEST_HAS_DEATH_TEST TEST(GeneratedMessageReflectionTest, UsageErrors) { diff --git a/third_party/protobuf/src/google/protobuf/generated_message_util.cc b/third_party/protobuf/src/google/protobuf/generated_message_util.cc index 76e547b..ac32150 100644 --- a/third_party/protobuf/src/google/protobuf/generated_message_util.cc +++ b/third_party/protobuf/src/google/protobuf/generated_message_util.cc @@ -49,7 +49,6 @@ double NaN() { const ::std::string kEmptyString; - } // namespace internal } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/generated_message_util.h b/third_party/protobuf/src/google/protobuf/generated_message_util.h index 77ae106..b2fb8f0b 100644 --- a/third_party/protobuf/src/google/protobuf/generated_message_util.h +++ b/third_party/protobuf/src/google/protobuf/generated_message_util.h @@ -41,16 +41,8 @@ #include <string> #include <google/protobuf/stubs/common.h> - - namespace google { namespace protobuf { - namespace io { - class CodedInputStream; // coded_stream.h - } -} - -namespace protobuf { namespace internal { // Annotation for the compiler to emit a deprecation message if a field marked @@ -60,11 +52,7 @@ namespace internal { // For internal use in the pb.cc files, deprecation warnings are suppressed // there. #undef DEPRECATED_PROTOBUF_FIELD -#if !defined(INTERNAL_SUPPRESS_PROTOBUF_FIELD_DEPRECATION) -# define PROTOBUF_DEPRECATED GOOGLE_ATTRIBUTE_DEPRECATED -#else -# define PROTOBUF_DEPRECATED -#endif +#define PROTOBUF_DEPRECATED // Constants for special floating point values. @@ -74,6 +62,13 @@ LIBPROTOBUF_EXPORT double NaN(); // Constant used for empty default strings. LIBPROTOBUF_EXPORT extern const ::std::string kEmptyString; +// Defined in generated_message_reflection.cc -- not actually part of the lite +// library. +// +// TODO(jasonh): The various callers get this declaration from a variety of +// places: probably in most cases repeated_field.h. Clean these up so they all +// get the declaration from this file. +LIBPROTOBUF_EXPORT int StringSpaceUsedExcludingSelf(const string& str); } // namespace internal } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/io/coded_stream.cc b/third_party/protobuf/src/google/protobuf/io/coded_stream.cc index 402a3ad..36add8c 100644 --- a/third_party/protobuf/src/google/protobuf/io/coded_stream.cc +++ b/third_party/protobuf/src/google/protobuf/io/coded_stream.cc @@ -43,7 +43,7 @@ #include <limits.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/stubs/common.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { @@ -69,6 +69,19 @@ inline bool NextNonEmpty(ZeroCopyInputStream* input, // CodedInputStream ================================================== +CodedInputStream::~CodedInputStream() { + if (input_ != NULL) { + BackUpInputToCurrentPosition(); + } + + if (total_bytes_warning_threshold_ == -2) { + GOOGLE_LOG(WARNING) << "The total number of bytes read was " << total_bytes_read_; + } +} + +// Static. +int CodedInputStream::default_recursion_limit_ = 100; + void CodedInputStream::BackUpInputToCurrentPosition() { int backup_bytes = BufferSize() + buffer_size_after_limit_ + overflow_bytes_; @@ -98,8 +111,7 @@ inline void CodedInputStream::RecomputeBufferLimits() { CodedInputStream::Limit CodedInputStream::PushLimit(int byte_limit) { // Current position relative to the beginning of the stream. - int current_position = total_bytes_read_ - - (BufferSize() + buffer_size_after_limit_); + int current_position = CurrentPosition(); Limit old_limit = current_limit_; @@ -133,10 +145,9 @@ void CodedInputStream::PopLimit(Limit limit) { legitimate_message_end_ = false; } -int CodedInputStream::BytesUntilLimit() { +int CodedInputStream::BytesUntilLimit() const { if (current_limit_ == INT_MAX) return -1; - int current_position = total_bytes_read_ - - (BufferSize() + buffer_size_after_limit_); + int current_position = CurrentPosition(); return current_limit_ - current_position; } @@ -145,10 +156,14 @@ void CodedInputStream::SetTotalBytesLimit( int total_bytes_limit, int warning_threshold) { // Make sure the limit isn't already past, since this could confuse other // code. - int current_position = total_bytes_read_ - - (BufferSize() + buffer_size_after_limit_); + int current_position = CurrentPosition(); total_bytes_limit_ = max(current_position, total_bytes_limit); - total_bytes_warning_threshold_ = warning_threshold; + if (warning_threshold >= 0) { + total_bytes_warning_threshold_ = warning_threshold; + } else { + // warning_threshold is negative + total_bytes_warning_threshold_ = -1; + } RecomputeBufferLimits(); } @@ -368,16 +383,17 @@ uint32 CodedInputStream::ReadTagSlow() { // For the slow path, just do a 64-bit read. Try to optimize for one-byte tags // again, since we have now refreshed the buffer. - uint64 result; + uint64 result = 0; if (!ReadVarint64(&result)) return 0; return static_cast<uint32>(result); } uint32 CodedInputStream::ReadTagFallback() { - if (BufferSize() >= kMaxVarintBytes || + const int buf_size = BufferSize(); + if (buf_size >= kMaxVarintBytes || // Optimization: If the varint ends at exactly the end of the buffer, // we can detect that and still use the fast path. - (buffer_end_ > buffer_ && !(buffer_end_[-1] & 0x80))) { + (buf_size > 0 && !(buffer_end_[-1] & 0x80))) { uint32 tag; const uint8* end = ReadVarint32FromArray(buffer_, &tag); if (end == NULL) { @@ -388,7 +404,9 @@ uint32 CodedInputStream::ReadTagFallback() { } else { // We are commonly at a limit when attempting to read tags. Try to quickly // detect this case without making another function call. - if (buffer_ == buffer_end_ && buffer_size_after_limit_ > 0 && + if ((buf_size == 0) && + ((buffer_size_after_limit_ > 0) || + (total_bytes_read_ == current_limit_)) && // Make sure that the limit we hit is not total_bytes_limit_, since // in that case we still need to call Refresh() so that it prints an // error. @@ -492,8 +510,8 @@ bool CodedInputStream::Refresh() { "CodedInputStream::SetTotalBytesLimit() in " "google/protobuf/io/coded_stream.h."; - // Don't warn again for this stream. - total_bytes_warning_threshold_ = -1; + // Don't warn again for this stream, and print total size at the end. + total_bytes_warning_threshold_ = -2; } const void* void_buffer; diff --git a/third_party/protobuf/src/google/protobuf/io/coded_stream.h b/third_party/protobuf/src/google/protobuf/io/coded_stream.h index 97ac507..66cbee0 100644 --- a/third_party/protobuf/src/google/protobuf/io/coded_stream.h +++ b/third_party/protobuf/src/google/protobuf/io/coded_stream.h @@ -170,6 +170,9 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // successfully and the stream's byte limit. ~CodedInputStream(); + // Return true if this CodedInputStream reads from a flat array instead of + // a ZeroCopyInputStream. + inline bool IsFlat() const; // Skips a number of bytes. Returns false if an underlying read error // occurs. @@ -311,7 +314,10 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // Returns the number of bytes left until the nearest limit on the // stack is hit, or -1 if no limits are in place. - int BytesUntilLimit(); + int BytesUntilLimit() const; + + // Returns current position relative to the beginning of the input stream. + int CurrentPosition() const; // Total Bytes Limit ----------------------------------------------- // To prevent malicious users from sending excessively large messages @@ -327,8 +333,9 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // cause integer overflows is 512MB. The default limit is 64MB. Apps // should set shorter limits if possible. If warning_threshold is not -1, // a warning will be printed to stderr after warning_threshold bytes are - // read. An error will always be printed to stderr if the limit is - // reached. + // read. For backwards compatibility all negative values get squached to -1, + // as other negative values might have special internal meanings. + // An error will always be printed to stderr if the limit is reached. // // This is unrelated to PushLimit()/PopLimit(). // @@ -355,9 +362,10 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // messages and groups. CodedInputStream keeps track of this because it // is the only object that is passed down the stack during parsing. - // Sets the maximum recursion depth. The default is 64. + // Sets the maximum recursion depth. The default is 100. void SetRecursionLimit(int limit); + // Increments the current recursion depth. Returns true if the depth is // under the limit, false if it has gone over. bool IncrementRecursionDepth(); @@ -433,7 +441,8 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // // Note that this feature is ignored when parsing "lite" messages as they do // not have descriptors. - void SetExtensionRegistry(DescriptorPool* pool, MessageFactory* factory); + void SetExtensionRegistry(const DescriptorPool* pool, + MessageFactory* factory); // Get the DescriptorPool set via SetExtensionRegistry(), or NULL if no pool // has been provided. @@ -482,6 +491,11 @@ class LIBPROTOBUF_EXPORT CodedInputStream { // Maximum number of bytes to read, period. This is unrelated to // current_limit_. Set using SetTotalBytesLimit(). int total_bytes_limit_; + + // If positive/0: Limit for bytes read after which a warning due to size + // should be logged. + // If -1: Printing of warning disabled. Can be set by client. + // If -2: Internal: Limit has been reached, print full size when destructing. int total_bytes_warning_threshold_; // Current recursion depth, controlled by IncrementRecursionDepth() and @@ -539,7 +553,8 @@ class LIBPROTOBUF_EXPORT CodedInputStream { static const int kDefaultTotalBytesLimit = 64 << 20; // 64MB static const int kDefaultTotalBytesWarningThreshold = 32 << 20; // 32MB - static const int kDefaultRecursionLimit = 64; + + static int default_recursion_limit_; // 100 by default. }; // Class which encodes and writes binary data which is composed of varint- @@ -891,7 +906,9 @@ inline bool CodedInputStream::ExpectAtEnd() { // If we are at a limit we know no more bytes can be read. Otherwise, it's // hard to say without calling Refresh(), and we'd rather not do that. - if (buffer_ == buffer_end_ && buffer_size_after_limit_ != 0) { + if (buffer_ == buffer_end_ && + ((buffer_size_after_limit_ != 0) || + (total_bytes_read_ == current_limit_))) { last_tag_ = 0; // Pretend we called ReadTag()... legitimate_message_end_ = true; // ... and it hit EOF. return true; @@ -900,6 +917,10 @@ inline bool CodedInputStream::ExpectAtEnd() { } } +inline int CodedInputStream::CurrentPosition() const { + return total_bytes_read_ - (BufferSize() + buffer_size_after_limit_); +} + inline uint8* CodedOutputStream::GetDirectBufferForNBytesAndAdvance(int size) { if (buffer_size_ < size) { return NULL; @@ -1039,7 +1060,7 @@ inline void CodedInputStream::DecrementRecursionDepth() { if (recursion_depth_ > 0) --recursion_depth_; } -inline void CodedInputStream::SetExtensionRegistry(DescriptorPool* pool, +inline void CodedInputStream::SetExtensionRegistry(const DescriptorPool* pool, MessageFactory* factory) { extension_pool_ = pool; extension_factory_ = factory; @@ -1071,7 +1092,7 @@ inline CodedInputStream::CodedInputStream(ZeroCopyInputStream* input) total_bytes_limit_(kDefaultTotalBytesLimit), total_bytes_warning_threshold_(kDefaultTotalBytesWarningThreshold), recursion_depth_(0), - recursion_limit_(kDefaultRecursionLimit), + recursion_limit_(default_recursion_limit_), extension_pool_(NULL), extension_factory_(NULL) { // Eagerly Refresh() so buffer space is immediately available. @@ -1092,17 +1113,15 @@ inline CodedInputStream::CodedInputStream(const uint8* buffer, int size) total_bytes_limit_(kDefaultTotalBytesLimit), total_bytes_warning_threshold_(kDefaultTotalBytesWarningThreshold), recursion_depth_(0), - recursion_limit_(kDefaultRecursionLimit), + recursion_limit_(default_recursion_limit_), extension_pool_(NULL), extension_factory_(NULL) { // Note that setting current_limit_ == size is important to prevent some // code paths from trying to access input_ and segfaulting. } -inline CodedInputStream::~CodedInputStream() { - if (input_ != NULL) { - BackUpInputToCurrentPosition(); - } +inline bool CodedInputStream::IsFlat() const { + return input_ == NULL; } } // namespace io diff --git a/third_party/protobuf/src/google/protobuf/io/coded_stream_inl.h b/third_party/protobuf/src/google/protobuf/io/coded_stream_inl.h index 8156602..144f44f 100644 --- a/third_party/protobuf/src/google/protobuf/io/coded_stream_inl.h +++ b/third_party/protobuf/src/google/protobuf/io/coded_stream_inl.h @@ -38,7 +38,7 @@ #include <google/protobuf/io/coded_stream.h> #include <string> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { diff --git a/third_party/protobuf/src/google/protobuf/io/coded_stream_unittest.cc b/third_party/protobuf/src/google/protobuf/io/coded_stream_unittest.cc index ff268ab..2daab19 100644 --- a/third_party/protobuf/src/google/protobuf/io/coded_stream_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/io/coded_stream_unittest.cc @@ -44,7 +44,6 @@ #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> #include <google/protobuf/io/zero_copy_stream_impl.h> -#include <google/protobuf/stubs/strutil.h> // This declares an unsigned long long integer literal in a portable way. @@ -125,6 +124,13 @@ namespace { class CodedStreamTest : public testing::Test { protected: + // Helper method used by tests for bytes warning. See implementation comment + // for further information. + static void SetupTotalBytesLimitWarningTest( + int total_bytes_limit, int warning_threshold, + vector<string>* out_errors, vector<string>* out_warnings); + + // Buffer used during most of the tests. This assumes tests run sequentially. static const int kBufferSize = 1024 * 64; static uint8 buffer_[kBufferSize]; }; @@ -1022,6 +1028,59 @@ TEST_F(CodedStreamTest, TotalBytesLimitNotValidMessageEnd) { EXPECT_FALSE(coded_input.ConsumedEntireMessage()); } +// This method is used by the tests below. +// It constructs a CodedInputStream with the given limits and tries to read 2KiB +// of data from it. Then it returns the logged errors and warnings in the given +// vectors. +void CodedStreamTest::SetupTotalBytesLimitWarningTest( + int total_bytes_limit, int warning_threshold, + vector<string>* out_errors, vector<string>* out_warnings) { + ArrayInputStream raw_input(buffer_, sizeof(buffer_), 128); + + ScopedMemoryLog scoped_log; + { + CodedInputStream input(&raw_input); + input.SetTotalBytesLimit(total_bytes_limit, warning_threshold); + string str; + EXPECT_TRUE(input.ReadString(&str, 2048)); + } + + *out_errors = scoped_log.GetMessages(ERROR); + *out_warnings = scoped_log.GetMessages(WARNING); +} + +TEST_F(CodedStreamTest, TotalBytesLimitWarning) { + vector<string> errors; + vector<string> warnings; + SetupTotalBytesLimitWarningTest(10240, 1024, &errors, &warnings); + + EXPECT_EQ(0, errors.size()); + + ASSERT_EQ(2, warnings.size()); + EXPECT_PRED_FORMAT2(testing::IsSubstring, + "Reading dangerously large protocol message. If the message turns out to " + "be larger than 10240 bytes, parsing will be halted for security reasons.", + warnings[0]); + EXPECT_PRED_FORMAT2(testing::IsSubstring, + "The total number of bytes read was 2048", + warnings[1]); +} + +TEST_F(CodedStreamTest, TotalBytesLimitWarningDisabled) { + vector<string> errors; + vector<string> warnings; + + // Test with -1 + SetupTotalBytesLimitWarningTest(10240, -1, &errors, &warnings); + EXPECT_EQ(0, errors.size()); + EXPECT_EQ(0, warnings.size()); + + // Test again with -2, expecting the same result + SetupTotalBytesLimitWarningTest(10240, -2, &errors, &warnings); + EXPECT_EQ(0, errors.size()); + EXPECT_EQ(0, warnings.size()); +} + TEST_F(CodedStreamTest, RecursionLimit) { ArrayInputStream input(buffer_, sizeof(buffer_)); @@ -1060,6 +1119,7 @@ TEST_F(CodedStreamTest, RecursionLimit) { EXPECT_FALSE(coded_input.IncrementRecursionDepth()); // 7 } + class ReallyBigInputStream : public ZeroCopyInputStream { public: ReallyBigInputStream() : backup_amount_(0), buffer_count_(0) {} diff --git a/third_party/protobuf/src/google/protobuf/io/gzip_stream.cc b/third_party/protobuf/src/google/protobuf/io/gzip_stream.cc index 0f1ff87..fe1f331 100644 --- a/third_party/protobuf/src/google/protobuf/io/gzip_stream.cc +++ b/third_party/protobuf/src/google/protobuf/io/gzip_stream.cc @@ -199,16 +199,6 @@ GzipOutputStream::GzipOutputStream(ZeroCopyOutputStream* sub_stream, Init(sub_stream, options); } -GzipOutputStream::GzipOutputStream( - ZeroCopyOutputStream* sub_stream, Format format, int buffer_size) { - Options options; - options.format = format; - if (buffer_size != -1) { - options.buffer_size = buffer_size; - } - Init(sub_stream, options); -} - void GzipOutputStream::Init(ZeroCopyOutputStream* sub_stream, const Options& options) { sub_stream_ = sub_stream; @@ -309,10 +299,11 @@ int64 GzipOutputStream::ByteCount() const { } bool GzipOutputStream::Flush() { - do { - zerror_ = Deflate(Z_FULL_FLUSH); - } while (zerror_ == Z_OK); - return zerror_ == Z_OK; + zerror_ = Deflate(Z_FULL_FLUSH); + // Return true if the flush succeeded or if it was a no-op. + return (zerror_ == Z_OK) || + (zerror_ == Z_BUF_ERROR && zcontext_.avail_in == 0 && + zcontext_.avail_out != 0); } bool GzipOutputStream::Close() { diff --git a/third_party/protobuf/src/google/protobuf/io/gzip_stream.h b/third_party/protobuf/src/google/protobuf/io/gzip_stream.h index 65dbc5b..7ee24bc 100644 --- a/third_party/protobuf/src/google/protobuf/io/gzip_stream.h +++ b/third_party/protobuf/src/google/protobuf/io/gzip_stream.h @@ -45,6 +45,7 @@ #include <zlib.h> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/io/zero_copy_stream.h> namespace google { @@ -144,12 +145,6 @@ class LIBPROTOBUF_EXPORT GzipOutputStream : public ZeroCopyOutputStream { ZeroCopyOutputStream* sub_stream, const Options& options); - // DEPRECATED: Use one of the above constructors instead. - GzipOutputStream( - ZeroCopyOutputStream* sub_stream, - Format format, - int buffer_size = -1) GOOGLE_ATTRIBUTE_DEPRECATED; - virtual ~GzipOutputStream(); // Return last error message or NULL if no error. @@ -165,6 +160,13 @@ class LIBPROTOBUF_EXPORT GzipOutputStream : public ZeroCopyOutputStream { // necessary. // Compression may be less efficient stopping and starting around flushes. // Returns true if no error. + // + // Please ensure that block size is > 6. Here is an excerpt from the zlib + // doc that explains why: + // + // In the case of a Z_FULL_FLUSH or Z_SYNC_FLUSH, make sure that avail_out + // is greater than six to avoid repeated flush markers due to + // avail_out == 0 on return. bool Flush(); // Writes out all data and closes the gzip stream. diff --git a/third_party/protobuf/src/google/protobuf/io/printer.cc b/third_party/protobuf/src/google/protobuf/io/printer.cc index 9ab90de..d2bf3f5 100644 --- a/third_party/protobuf/src/google/protobuf/io/printer.cc +++ b/third_party/protobuf/src/google/protobuf/io/printer.cc @@ -35,7 +35,6 @@ #include <google/protobuf/io/printer.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/stubs/common.h> -#include <google/protobuf/stubs/strutil.h> namespace google { namespace protobuf { @@ -51,8 +50,8 @@ Printer::Printer(ZeroCopyOutputStream* output, char variable_delimiter) } Printer::~Printer() { - // Only BackUp() if we're sure we've successfully called Next() at least once. - if (buffer_size_ > 0) { + // Only BackUp() if we have called Next() at least once and never failed. + if (buffer_size_ > 0 && !failed_) { output_->BackUp(buffer_size_); } } @@ -169,7 +168,7 @@ void Printer::WriteRaw(const char* data, int size) { if (failed_) return; if (size == 0) return; - if (at_start_of_line_) { + if (at_start_of_line_ && (size > 0) && (data[0] != '\n')) { // Insert an indent. at_start_of_line_ = false; WriteRaw(indent_.data(), indent_.size()); diff --git a/third_party/protobuf/src/google/protobuf/io/printer_unittest.cc b/third_party/protobuf/src/google/protobuf/io/printer_unittest.cc index 580a53d..399395c 100644 --- a/third_party/protobuf/src/google/protobuf/io/printer_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/io/printer_unittest.cc @@ -233,7 +233,31 @@ TEST(Printer, Death) { } #endif // GTEST_HAS_DEATH_TEST -TEST(Printer, WriteFailure) { +TEST(Printer, WriteFailurePartial) { + char buffer[17]; + + ArrayOutputStream output(buffer, sizeof(buffer)); + Printer printer(&output, '$'); + + // Print 16 bytes to almost fill the buffer (should not fail). + printer.Print("0123456789abcdef"); + EXPECT_FALSE(printer.failed()); + + // Try to print 2 chars. Only one fits. + printer.Print("<>"); + EXPECT_TRUE(printer.failed()); + + // Anything else should fail too. + printer.Print(" "); + EXPECT_TRUE(printer.failed()); + printer.Print("blah"); + EXPECT_TRUE(printer.failed()); + + // Buffer should contain the first 17 bytes written. + EXPECT_EQ("0123456789abcdef<", string(buffer, sizeof(buffer))); +} + +TEST(Printer, WriteFailureExact) { char buffer[16]; ArrayOutputStream output(buffer, sizeof(buffer)); diff --git a/third_party/protobuf/src/google/protobuf/io/tokenizer.cc b/third_party/protobuf/src/google/protobuf/io/tokenizer.cc index 513831d..a022b71 100644 --- a/third_party/protobuf/src/google/protobuf/io/tokenizer.cc +++ b/third_party/protobuf/src/google/protobuf/io/tokenizer.cc @@ -89,8 +89,11 @@ // exactly pretty. #include <google/protobuf/io/tokenizer.h> +#include <google/protobuf/stubs/common.h> +#include <google/protobuf/stubs/stringprintf.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/stubs/strutil.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -118,6 +121,8 @@ namespace { CHARACTER_CLASS(Whitespace, c == ' ' || c == '\n' || c == '\t' || c == '\r' || c == '\v' || c == '\f'); +CHARACTER_CLASS(WhitespaceNoNewline, c == ' ' || c == '\t' || + c == '\r' || c == '\v' || c == '\f'); CHARACTER_CLASS(Unprintable, c < ' ' && c > '\0'); @@ -187,7 +192,8 @@ Tokenizer::Tokenizer(ZeroCopyInputStream* input, read_error_(false), line_(0), column_(0), - token_start_(-1), + record_target_(NULL), + record_start_(-1), allow_f_after_float_(false), comment_style_(CPP_COMMENT_STYLE) { @@ -238,9 +244,9 @@ void Tokenizer::Refresh() { } // If we're in a token, append the rest of the buffer to it. - if (token_start_ >= 0 && token_start_ < buffer_size_) { - current_.text.append(buffer_ + token_start_, buffer_size_ - token_start_); - token_start_ = 0; + if (record_target_ != NULL && record_start_ < buffer_size_) { + record_target_->append(buffer_ + record_start_, buffer_size_ - record_start_); + record_start_ = 0; } const void* data = NULL; @@ -261,23 +267,33 @@ void Tokenizer::Refresh() { current_char_ = buffer_[0]; } +inline void Tokenizer::RecordTo(string* target) { + record_target_ = target; + record_start_ = buffer_pos_; +} + +inline void Tokenizer::StopRecording() { + // Note: The if() is necessary because some STL implementations crash when + // you call string::append(NULL, 0), presumably because they are trying to + // be helpful by detecting the NULL pointer, even though there's nothing + // wrong with reading zero bytes from NULL. + if (buffer_pos_ != record_start_) { + record_target_->append(buffer_ + record_start_, buffer_pos_ - record_start_); + } + record_target_ = NULL; + record_start_ = -1; +} + inline void Tokenizer::StartToken() { - token_start_ = buffer_pos_; current_.type = TYPE_START; // Just for the sake of initializing it. current_.text.clear(); current_.line = line_; current_.column = column_; + RecordTo(¤t_.text); } inline void Tokenizer::EndToken() { - // Note: The if() is necessary because some STL implementations crash when - // you call string::append(NULL, 0), presumably because they are trying to - // be helpful by detecting the NULL pointer, even though there's nothing - // wrong with reading zero bytes from NULL. - if (buffer_pos_ != token_start_) { - current_.text.append(buffer_ + token_start_, buffer_pos_ - token_start_); - } - token_start_ = -1; + StopRecording(); current_.end_column = column_; } @@ -353,6 +369,27 @@ void Tokenizer::ConsumeString(char delimiter) { AddError("Expected hex digits for escape sequence."); } // Possibly followed by another hex digit, but again we don't care. + } else if (TryConsume('u')) { + if (!TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>()) { + AddError("Expected four hex digits for \\u escape sequence."); + } + } else if (TryConsume('U')) { + // We expect 8 hex digits; but only the range up to 0x10ffff is + // legal. + if (!TryConsume('0') || + !TryConsume('0') || + !(TryConsume('0') || TryConsume('1')) || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>() || + !TryConsumeOne<HexDigit>()) { + AddError("Expected eight hex digits up to 10ffff for \\U escape " + "sequence"); + } } else { AddError("Invalid escape sequence in string literal."); } @@ -426,26 +463,51 @@ Tokenizer::TokenType Tokenizer::ConsumeNumber(bool started_with_zero, return is_float ? TYPE_FLOAT : TYPE_INTEGER; } -void Tokenizer::ConsumeLineComment() { +void Tokenizer::ConsumeLineComment(string* content) { + if (content != NULL) RecordTo(content); + while (current_char_ != '\0' && current_char_ != '\n') { NextChar(); } TryConsume('\n'); + + if (content != NULL) StopRecording(); } -void Tokenizer::ConsumeBlockComment() { +void Tokenizer::ConsumeBlockComment(string* content) { int start_line = line_; int start_column = column_ - 2; + if (content != NULL) RecordTo(content); + while (true) { while (current_char_ != '\0' && current_char_ != '*' && - current_char_ != '/') { + current_char_ != '/' && + current_char_ != '\n') { NextChar(); } - if (TryConsume('*') && TryConsume('/')) { + if (TryConsume('\n')) { + if (content != NULL) StopRecording(); + + // Consume leading whitespace and asterisk; + ConsumeZeroOrMore<WhitespaceNoNewline>(); + if (TryConsume('*')) { + if (TryConsume('/')) { + // End of comment. + break; + } + } + + if (content != NULL) RecordTo(content); + } else if (TryConsume('*') && TryConsume('/')) { // End of comment. + if (content != NULL) { + StopRecording(); + // Strip trailing "*/". + content->erase(content->size() - 2); + } break; } else if (TryConsume('/') && current_char_ == '*') { // Note: We didn't consume the '*' because if there is a '/' after it @@ -456,42 +518,59 @@ void Tokenizer::ConsumeBlockComment() { AddError("End-of-file inside block comment."); error_collector_->AddError( start_line, start_column, " Comment started here."); + if (content != NULL) StopRecording(); break; } } } +Tokenizer::NextCommentStatus Tokenizer::TryConsumeCommentStart() { + if (comment_style_ == CPP_COMMENT_STYLE && TryConsume('/')) { + if (TryConsume('/')) { + return LINE_COMMENT; + } else if (TryConsume('*')) { + return BLOCK_COMMENT; + } else { + // Oops, it was just a slash. Return it. + current_.type = TYPE_SYMBOL; + current_.text = "/"; + current_.line = line_; + current_.column = column_ - 1; + current_.end_column = column_; + return SLASH_NOT_COMMENT; + } + } else if (comment_style_ == SH_COMMENT_STYLE && TryConsume('#')) { + return LINE_COMMENT; + } else { + return NO_COMMENT; + } +} + // ------------------------------------------------------------------- bool Tokenizer::Next() { previous_ = current_; - // Did we skip any characters after the last token? - bool skipped_stuff = false; - while (!read_error_) { - if (TryConsumeOne<Whitespace>()) { - ConsumeZeroOrMore<Whitespace>(); - - } else if (comment_style_ == CPP_COMMENT_STYLE && TryConsume('/')) { - // Starting a comment? - if (TryConsume('/')) { - ConsumeLineComment(); - } else if (TryConsume('*')) { - ConsumeBlockComment(); - } else { - // Oops, it was just a slash. Return it. - current_.type = TYPE_SYMBOL; - current_.text = "/"; - current_.line = line_; - current_.column = column_ - 1; + ConsumeZeroOrMore<Whitespace>(); + + switch (TryConsumeCommentStart()) { + case LINE_COMMENT: + ConsumeLineComment(NULL); + continue; + case BLOCK_COMMENT: + ConsumeBlockComment(NULL); + continue; + case SLASH_NOT_COMMENT: return true; - } + case NO_COMMENT: + break; + } - } else if (comment_style_ == SH_COMMENT_STYLE && TryConsume('#')) { - ConsumeLineComment(); + // Check for EOF before continuing. + if (read_error_) break; - } else if (LookingAt<Unprintable>() || current_char_ == '\0') { + if (LookingAt<Unprintable>() || current_char_ == '\0') { AddError("Invalid control characters encountered in text."); NextChar(); // Skip more unprintable characters, too. But, remember that '\0' is @@ -519,7 +598,9 @@ bool Tokenizer::Next() { if (TryConsumeOne<Digit>()) { // It's a floating-point number. - if (previous_.type == TYPE_IDENTIFIER && !skipped_stuff) { + if (previous_.type == TYPE_IDENTIFIER && + current_.line == previous_.line && + current_.column == previous_.end_column) { // We don't accept syntax like "blah.123". error_collector_->AddError(line_, column_ - 2, "Need space between identifier and decimal point."); @@ -544,8 +625,6 @@ bool Tokenizer::Next() { EndToken(); return true; } - - skipped_stuff = true; } // EOF @@ -557,6 +636,195 @@ bool Tokenizer::Next() { return false; } +namespace { + +// Helper class for collecting comments and putting them in the right places. +// +// This basically just buffers the most recent comment until it can be decided +// exactly where that comment should be placed. When Flush() is called, the +// current comment goes into either prev_trailing_comments or detached_comments. +// When the CommentCollector is destroyed, the last buffered comment goes into +// next_leading_comments. +class CommentCollector { + public: + CommentCollector(string* prev_trailing_comments, + vector<string>* detached_comments, + string* next_leading_comments) + : prev_trailing_comments_(prev_trailing_comments), + detached_comments_(detached_comments), + next_leading_comments_(next_leading_comments), + has_comment_(false), + is_line_comment_(false), + can_attach_to_prev_(true) { + if (prev_trailing_comments != NULL) prev_trailing_comments->clear(); + if (detached_comments != NULL) detached_comments->clear(); + if (next_leading_comments != NULL) next_leading_comments->clear(); + } + + ~CommentCollector() { + // Whatever is in the buffer is a leading comment. + if (next_leading_comments_ != NULL && has_comment_) { + comment_buffer_.swap(*next_leading_comments_); + } + } + + // About to read a line comment. Get the comment buffer pointer in order to + // read into it. + string* GetBufferForLineComment() { + // We want to combine with previous line comments, but not block comments. + if (has_comment_ && !is_line_comment_) { + Flush(); + } + has_comment_ = true; + is_line_comment_ = true; + return &comment_buffer_; + } + + // About to read a block comment. Get the comment buffer pointer in order to + // read into it. + string* GetBufferForBlockComment() { + if (has_comment_) { + Flush(); + } + has_comment_ = true; + is_line_comment_ = false; + return &comment_buffer_; + } + + void ClearBuffer() { + comment_buffer_.clear(); + has_comment_ = false; + } + + // Called once we know that the comment buffer is complete and is *not* + // connected to the next token. + void Flush() { + if (has_comment_) { + if (can_attach_to_prev_) { + if (prev_trailing_comments_ != NULL) { + prev_trailing_comments_->append(comment_buffer_); + } + can_attach_to_prev_ = false; + } else { + if (detached_comments_ != NULL) { + detached_comments_->push_back(comment_buffer_); + } + } + ClearBuffer(); + } + } + + void DetachFromPrev() { + can_attach_to_prev_ = false; + } + + private: + string* prev_trailing_comments_; + vector<string>* detached_comments_; + string* next_leading_comments_; + + string comment_buffer_; + + // True if any comments were read into comment_buffer_. This can be true even + // if comment_buffer_ is empty, namely if the comment was "/**/". + bool has_comment_; + + // Is the comment in the comment buffer a line comment? + bool is_line_comment_; + + // Is it still possible that we could be reading a comment attached to the + // previous token? + bool can_attach_to_prev_; +}; + +} // namespace + +bool Tokenizer::NextWithComments(string* prev_trailing_comments, + vector<string>* detached_comments, + string* next_leading_comments) { + CommentCollector collector(prev_trailing_comments, detached_comments, + next_leading_comments); + + if (current_.type == TYPE_START) { + collector.DetachFromPrev(); + } else { + // A comment appearing on the same line must be attached to the previous + // declaration. + ConsumeZeroOrMore<WhitespaceNoNewline>(); + switch (TryConsumeCommentStart()) { + case LINE_COMMENT: + ConsumeLineComment(collector.GetBufferForLineComment()); + + // Don't allow comments on subsequent lines to be attached to a trailing + // comment. + collector.Flush(); + break; + case BLOCK_COMMENT: + ConsumeBlockComment(collector.GetBufferForBlockComment()); + + ConsumeZeroOrMore<WhitespaceNoNewline>(); + if (!TryConsume('\n')) { + // Oops, the next token is on the same line. If we recorded a comment + // we really have no idea which token it should be attached to. + collector.ClearBuffer(); + return Next(); + } + + // Don't allow comments on subsequent lines to be attached to a trailing + // comment. + collector.Flush(); + break; + case SLASH_NOT_COMMENT: + return true; + case NO_COMMENT: + if (!TryConsume('\n')) { + // The next token is on the same line. There are no comments. + return Next(); + } + break; + } + } + + // OK, we are now on the line *after* the previous token. + while (true) { + ConsumeZeroOrMore<WhitespaceNoNewline>(); + + switch (TryConsumeCommentStart()) { + case LINE_COMMENT: + ConsumeLineComment(collector.GetBufferForLineComment()); + break; + case BLOCK_COMMENT: + ConsumeBlockComment(collector.GetBufferForBlockComment()); + + // Consume the rest of the line so that we don't interpret it as a + // blank line the next time around the loop. + ConsumeZeroOrMore<WhitespaceNoNewline>(); + TryConsume('\n'); + break; + case SLASH_NOT_COMMENT: + return true; + case NO_COMMENT: + if (TryConsume('\n')) { + // Completely blank line. + collector.Flush(); + collector.DetachFromPrev(); + } else { + bool result = Next(); + if (!result || + current_.text == "}" || + current_.text == "]" || + current_.text == ")") { + // It looks like we're at the end of a scope. In this case it + // makes no sense to attach a comment to the following token. + collector.Flush(); + } + return result; + } + break; + } + } +} + // ------------------------------------------------------------------- // Token-parsing helpers. Remember that these don't need to report // errors since any errors should already have been reported while @@ -626,17 +894,138 @@ double Tokenizer::ParseFloat(const string& text) { return result; } +// Helper to append a Unicode code point to a string as UTF8, without bringing +// in any external dependencies. +static void AppendUTF8(uint32 code_point, string* output) { + uint32 tmp = 0; + int len = 0; + if (code_point <= 0x7f) { + tmp = code_point; + len = 1; + } else if (code_point <= 0x07ff) { + tmp = 0x0000c080 | + ((code_point & 0x07c0) << 2) | + (code_point & 0x003f); + len = 2; + } else if (code_point <= 0xffff) { + tmp = 0x00e08080 | + ((code_point & 0xf000) << 4) | + ((code_point & 0x0fc0) << 2) | + (code_point & 0x003f); + len = 3; + } else if (code_point <= 0x1fffff) { + tmp = 0xf0808080 | + ((code_point & 0x1c0000) << 6) | + ((code_point & 0x03f000) << 4) | + ((code_point & 0x000fc0) << 2) | + (code_point & 0x003f); + len = 4; + } else { + // UTF-16 is only defined for code points up to 0x10FFFF, and UTF-8 is + // normally only defined up to there as well. + StringAppendF(output, "\\U%08x", code_point); + return; + } + tmp = ghtonl(tmp); + output->append(reinterpret_cast<const char*>(&tmp) + sizeof(tmp) - len, len); +} + +// Try to read <len> hex digits from ptr, and stuff the numeric result into +// *result. Returns true if that many digits were successfully consumed. +static bool ReadHexDigits(const char* ptr, int len, uint32* result) { + *result = 0; + if (len == 0) return false; + for (const char* end = ptr + len; ptr < end; ++ptr) { + if (*ptr == '\0') return false; + *result = (*result << 4) + DigitValue(*ptr); + } + return true; +} + +// Handling UTF-16 surrogate pairs. UTF-16 encodes code points in the range +// 0x10000...0x10ffff as a pair of numbers, a head surrogate followed by a trail +// surrogate. These numbers are in a reserved range of Unicode code points, so +// if we encounter such a pair we know how to parse it and convert it into a +// single code point. +static const uint32 kMinHeadSurrogate = 0xd800; +static const uint32 kMaxHeadSurrogate = 0xdc00; +static const uint32 kMinTrailSurrogate = 0xdc00; +static const uint32 kMaxTrailSurrogate = 0xe000; + +static inline bool IsHeadSurrogate(uint32 code_point) { + return (code_point >= kMinHeadSurrogate) && (code_point < kMaxHeadSurrogate); +} + +static inline bool IsTrailSurrogate(uint32 code_point) { + return (code_point >= kMinTrailSurrogate) && + (code_point < kMaxTrailSurrogate); +} + +// Combine a head and trail surrogate into a single Unicode code point. +static uint32 AssembleUTF16(uint32 head_surrogate, uint32 trail_surrogate) { + GOOGLE_DCHECK(IsHeadSurrogate(head_surrogate)); + GOOGLE_DCHECK(IsTrailSurrogate(trail_surrogate)); + return 0x10000 + (((head_surrogate - kMinHeadSurrogate) << 10) | + (trail_surrogate - kMinTrailSurrogate)); +} + +// Convert the escape sequence parameter to a number of expected hex digits. +static inline int UnicodeLength(char key) { + if (key == 'u') return 4; + if (key == 'U') return 8; + return 0; +} + +// Given a pointer to the 'u' or 'U' starting a Unicode escape sequence, attempt +// to parse that sequence. On success, returns a pointer to the first char +// beyond that sequence, and fills in *code_point. On failure, returns ptr +// itself. +static const char* FetchUnicodePoint(const char* ptr, uint32* code_point) { + const char* p = ptr; + // Fetch the code point. + const int len = UnicodeLength(*p++); + if (!ReadHexDigits(p, len, code_point)) + return ptr; + p += len; + + // Check if the code point we read is a "head surrogate." If so, then we + // expect it to be immediately followed by another code point which is a valid + // "trail surrogate," and together they form a UTF-16 pair which decodes into + // a single Unicode point. Trail surrogates may only use \u, not \U. + if (IsHeadSurrogate(*code_point) && *p == '\\' && *(p + 1) == 'u') { + uint32 trail_surrogate; + if (ReadHexDigits(p + 2, 4, &trail_surrogate) && + IsTrailSurrogate(trail_surrogate)) { + *code_point = AssembleUTF16(*code_point, trail_surrogate); + p += 6; + } + // If this failed, then we just emit the head surrogate as a code point. + // It's bogus, but so is the string. + } + + return p; +} + +// The text string must begin and end with single or double quote +// characters. void Tokenizer::ParseStringAppend(const string& text, string* output) { - // Reminder: text[0] is always the quote character. (If text is - // empty, it's invalid, so we'll just return.) - if (text.empty()) { + // Reminder: text[0] is always a quote character. (If text is + // empty, it's invalid, so we'll just return). + const size_t text_size = text.size(); + if (text_size == 0) { GOOGLE_LOG(DFATAL) << " Tokenizer::ParseStringAppend() passed text that could not" " have been tokenized as a string: " << CEscape(text); return; } - output->reserve(output->size() + text.size()); + // Reserve room for new string. The branch is necessary because if + // there is already space available the reserve() call might + // downsize the output. + const size_t new_len = text_size + output->size(); + if (new_len > output->capacity()) { + output->reserve(new_len); + } // Loop through the string copying characters to "output" and // interpreting escape sequences. Note that any invalid escape @@ -674,19 +1063,27 @@ void Tokenizer::ParseStringAppend(const string& text, string* output) { } output->push_back(static_cast<char>(code)); + } else if (*ptr == 'u' || *ptr == 'U') { + uint32 unicode; + const char* end = FetchUnicodePoint(ptr, &unicode); + if (end == ptr) { + // Failure: Just dump out what we saw, don't try to parse it. + output->push_back(*ptr); + } else { + AppendUTF8(unicode, output); + ptr = end - 1; // Because we're about to ++ptr. + } } else { // Some other escape code. output->push_back(TranslateEscape(*ptr)); } - } else if (*ptr == text[0]) { - // Ignore quote matching the starting quote. + } else if (*ptr == text[0] && ptr[1] == '\0') { + // Ignore final quote matching the starting quote. } else { output->push_back(*ptr); } } - - return; } } // namespace io diff --git a/third_party/protobuf/src/google/protobuf/io/tokenizer.h b/third_party/protobuf/src/google/protobuf/io/tokenizer.h index 8f759ab..d85b82f 100644 --- a/third_party/protobuf/src/google/protobuf/io/tokenizer.h +++ b/third_party/protobuf/src/google/protobuf/io/tokenizer.h @@ -38,6 +38,7 @@ #define GOOGLE_PROTOBUF_IO_TOKENIZER_H__ #include <string> +#include <vector> #include <google/protobuf/stubs/common.h> namespace google { @@ -137,6 +138,53 @@ class LIBPROTOBUF_EXPORT Tokenizer { // reached. bool Next(); + // Like Next(), but also collects comments which appear between the previous + // and next tokens. + // + // Comments which appear to be attached to the previous token are stored + // in *prev_tailing_comments. Comments which appear to be attached to the + // next token are stored in *next_leading_comments. Comments appearing in + // between which do not appear to be attached to either will be added to + // detached_comments. Any of these parameters can be NULL to simply discard + // the comments. + // + // A series of line comments appearing on consecutive lines, with no other + // tokens appearing on those lines, will be treated as a single comment. + // + // Only the comment content is returned; comment markers (e.g. //) are + // stripped out. For block comments, leading whitespace and an asterisk will + // be stripped from the beginning of each line other than the first. Newlines + // are included in the output. + // + // Examples: + // + // optional int32 foo = 1; // Comment attached to foo. + // // Comment attached to bar. + // optional int32 bar = 2; + // + // optional string baz = 3; + // // Comment attached to baz. + // // Another line attached to baz. + // + // // Comment attached to qux. + // // + // // Another line attached to qux. + // optional double qux = 4; + // + // // Detached comment. This is not attached to qux or corge + // // because there are blank lines separating it from both. + // + // optional string corge = 5; + // /* Block comment attached + // * to corge. Leading asterisks + // * will be removed. */ + // /* Block comment attached to + // * grault. */ + // optional int32 grault = 6; + bool NextWithComments(string* prev_trailing_comments, + vector<string>* detached_comments, + string* next_leading_comments); + // Parse helpers --------------------------------------------------- // Parses a TYPE_FLOAT token. This never fails, so long as the text actually @@ -200,11 +248,12 @@ class LIBPROTOBUF_EXPORT Tokenizer { int line_; int column_; - // Position in buffer_ where StartToken() was called. If the token - // started in the previous buffer, this is zero, and current_.text already - // contains the part of the token from the previous buffer. If not - // currently parsing a token, this is -1. - int token_start_; + // String to which text should be appended as we advance through it. + // Call RecordTo(&str) to start recording and StopRecording() to stop. + // E.g. StartToken() calls RecordTo(¤t_.text). record_start_ is the + // position within the current buffer where recording started. + string* record_target_; + int record_start_; // Options. bool allow_f_after_float_; @@ -223,6 +272,9 @@ class LIBPROTOBUF_EXPORT Tokenizer { // Read a new buffer from the input. void Refresh(); + inline void RecordTo(string* target); + inline void StopRecording(); + // Called when the current character is the first character of a new // token (not including whitespace or comments). inline void StartToken(); @@ -255,9 +307,28 @@ class LIBPROTOBUF_EXPORT Tokenizer { TokenType ConsumeNumber(bool started_with_zero, bool started_with_dot); // Consume the rest of a line. - void ConsumeLineComment(); + void ConsumeLineComment(string* content); // Consume until "*/". - void ConsumeBlockComment(); + void ConsumeBlockComment(string* content); + + enum NextCommentStatus { + // Started a line comment. + LINE_COMMENT, + + // Started a block comment. + BLOCK_COMMENT, + + // Consumed a slash, then realized it wasn't a comment. current_ has + // been filled in with a slash token. The caller should return it. + SLASH_NOT_COMMENT, + + // We do not appear to be starting a comment here. + NO_COMMENT + }; + + // If we're at the start of a new comment, consume it and return what kind + // of comment it is. + NextCommentStatus TryConsumeCommentStart(); // ----------------------------------------------------------------- // These helper methods make the parsing code more readable. The diff --git a/third_party/protobuf/src/google/protobuf/io/tokenizer_unittest.cc b/third_party/protobuf/src/google/protobuf/io/tokenizer_unittest.cc index 106d080..8de4393 100644 --- a/third_party/protobuf/src/google/protobuf/io/tokenizer_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/io/tokenizer_unittest.cc @@ -32,9 +32,10 @@ // Based on original Protocol Buffers design by // Sanjay Ghemawat, Jeff Dean, and others. -#include <vector> -#include <math.h> #include <limits.h> +#include <math.h> + +#include <vector> #include <google/protobuf/io/tokenizer.h> #include <google/protobuf/io/zero_copy_stream_impl.h> @@ -514,6 +515,217 @@ TEST_1D(TokenizerTest, ShCommentStyle, kBlockSizes) { // ------------------------------------------------------------------- +// In each case, the input is expected to have two tokens named "prev" and +// "next" with comments in between. +struct DocCommentCase { + string input; + + const char* prev_trailing_comments; + const char* detached_comments[10]; + const char* next_leading_comments; +}; + +inline ostream& operator<<(ostream& out, + const DocCommentCase& test_case) { + return out << CEscape(test_case.input); +} + +DocCommentCase kDocCommentCases[] = { + { + "prev next", + + "", + {}, + "" + }, + + { + "prev /* ignored */ next", + + "", + {}, + "" + }, + + { + "prev // trailing comment\n" + "next", + + " trailing comment\n", + {}, + "" + }, + + { + "prev\n" + "// leading comment\n" + "// line 2\n" + "next", + + "", + {}, + " leading comment\n" + " line 2\n" + }, + + { + "prev\n" + "// trailing comment\n" + "// line 2\n" + "\n" + "next", + + " trailing comment\n" + " line 2\n", + {}, + "" + }, + + { + "prev // trailing comment\n" + "// leading comment\n" + "// line 2\n" + "next", + + " trailing comment\n", + {}, + " leading comment\n" + " line 2\n" + }, + + { + "prev /* trailing block comment */\n" + "/* leading block comment\n" + " * line 2\n" + " * line 3 */" + "next", + + " trailing block comment ", + {}, + " leading block comment\n" + " line 2\n" + " line 3 " + }, + + { + "prev\n" + "/* trailing block comment\n" + " * line 2\n" + " * line 3\n" + " */\n" + "/* leading block comment\n" + " * line 2\n" + " * line 3 */" + "next", + + " trailing block comment\n" + " line 2\n" + " line 3\n", + {}, + " leading block comment\n" + " line 2\n" + " line 3 " + }, + + { + "prev\n" + "// trailing comment\n" + "\n" + "// detached comment\n" + "// line 2\n" + "\n" + "// second detached comment\n" + "/* third detached comment\n" + " * line 2 */\n" + "// leading comment\n" + "next", + + " trailing comment\n", + { + " detached comment\n" + " line 2\n", + " second detached comment\n", + " third detached comment\n" + " line 2 " + }, + " leading comment\n" + }, + + { + "prev /**/\n" + "\n" + "// detached comment\n" + "\n" + "// leading comment\n" + "next", + + "", + { + " detached comment\n" + }, + " leading comment\n" + }, + + { + "prev /**/\n" + "// leading comment\n" + "next", + + "", + {}, + " leading comment\n" + }, + }; + +TEST_2D(TokenizerTest, DocComments, kDocCommentCases, kBlockSizes) { + // Set up the tokenizer. + TestInputStream input(kDocCommentCases_case.input.data(), + kDocCommentCases_case.input.size(), + kBlockSizes_case); + TestErrorCollector error_collector; + Tokenizer tokenizer(&input, &error_collector); + + // Set up a second tokenizer where we'll pass all NULLs to NextWithComments(). + TestInputStream input2(kDocCommentCases_case.input.data(), + kDocCommentCases_case.input.size(), + kBlockSizes_case); + Tokenizer tokenizer2(&input2, &error_collector); + + tokenizer.Next(); + tokenizer2.Next(); + + EXPECT_EQ("prev", tokenizer.current().text); + EXPECT_EQ("prev", tokenizer2.current().text); + + string prev_trailing_comments; + vector<string> detached_comments; + string next_leading_comments; + tokenizer.NextWithComments(&prev_trailing_comments, &detached_comments, + &next_leading_comments); + tokenizer2.NextWithComments(NULL, NULL, NULL); + EXPECT_EQ("next", tokenizer.current().text); + EXPECT_EQ("next", tokenizer2.current().text); + + EXPECT_EQ(kDocCommentCases_case.prev_trailing_comments, + prev_trailing_comments); + + for (int i = 0; i < detached_comments.size(); i++) { + ASSERT_LT(i, GOOGLE_ARRAYSIZE(kDocCommentCases)); + ASSERT_TRUE(kDocCommentCases_case.detached_comments[i] != NULL); + EXPECT_EQ(kDocCommentCases_case.detached_comments[i], + detached_comments[i]); + } + + // Verify that we matched all the detached comments. + EXPECT_EQ(NULL, + kDocCommentCases_case.detached_comments[detached_comments.size()]); + + EXPECT_EQ(kDocCommentCases_case.next_leading_comments, + next_leading_comments); +} + +// ------------------------------------------------------------------- + // Test parse helpers. It's not really worth setting up a full data-driven // test here. TEST_F(TokenizerTest, ParseInteger) { @@ -614,6 +826,22 @@ TEST_F(TokenizerTest, ParseString) { Tokenizer::ParseString("'\\", &output); EXPECT_EQ("\\", output); + // Experiment with Unicode escapes. Here are one-, two- and three-byte Unicode + // characters. + Tokenizer::ParseString("'\\u0024\\u00a2\\u20ac\\U00024b62XX'", &output); + EXPECT_EQ("$¢€ð¤¢XX", output); + // Same thing encoded using UTF16. + Tokenizer::ParseString("'\\u0024\\u00a2\\u20ac\\ud852\\udf62XX'", &output); + EXPECT_EQ("$¢€ð¤¢XX", output); + // Here's some broken UTF16; there's a head surrogate with no tail surrogate. + // We just output this as if it were UTF8; it's not a defined code point, but + // it has a defined encoding. + Tokenizer::ParseString("'\\ud852XX'", &output); + EXPECT_EQ("\xed\xa1\x92XX", output); + // Malformed escape: Demons may fly out of the nose. + Tokenizer::ParseString("\\u0", &output); + EXPECT_EQ("u0", output); + // Test invalid strings that will never be tokenized as strings. #ifdef GTEST_HAS_DEATH_TEST // death tests do not work on Windows yet EXPECT_DEBUG_DEATH(Tokenizer::ParseString("", &output), @@ -658,6 +886,12 @@ ErrorCase kErrorCases[] = { "0:4: String literals cannot cross line boundaries.\n" }, { "'bar\nfoo", true, "0:4: String literals cannot cross line boundaries.\n" }, + { "'\\u01' foo", true, + "0:5: Expected four hex digits for \\u escape sequence.\n" }, + { "'\\u01' foo", true, + "0:5: Expected four hex digits for \\u escape sequence.\n" }, + { "'\\uXYZ' foo", true, + "0:3: Expected four hex digits for \\u escape sequence.\n" }, // Integer errors. { "123foo", true, @@ -734,7 +968,7 @@ TEST_2D(TokenizerTest, Errors, kErrorCases, kBlockSizes) { } // Check that the errors match what was expected. - EXPECT_EQ(error_collector.text_, kErrorCases_case.errors); + EXPECT_EQ(kErrorCases_case.errors, error_collector.text_); // If the error was recoverable, make sure we saw "foo" after it. if (kErrorCases_case.recoverable) { @@ -760,6 +994,7 @@ TEST_1D(TokenizerTest, BackUpOnDestruction, kBlockSizes) { EXPECT_EQ(strlen("foo"), input.ByteCount()); } + } // namespace } // namespace io } // namespace protobuf diff --git a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl.cc b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl.cc index 1384c74..9fcbb62 100644 --- a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl.cc +++ b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl.cc @@ -46,7 +46,8 @@ #include <google/protobuf/io/zero_copy_stream_impl.h> #include <google/protobuf/stubs/common.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> + namespace google { namespace protobuf { diff --git a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc index e801251..f552e1f 100644 --- a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc +++ b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc @@ -32,9 +32,9 @@ // Based on original Protocol Buffers design by // Sanjay Ghemawat, Jeff Dean, and others. -#include <google/protobuf/io/zero_copy_stream_impl.h> +#include <google/protobuf/io/zero_copy_stream_impl_lite.h> #include <google/protobuf/stubs/common.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { diff --git a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_unittest.cc b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_unittest.cc index 5196d90..6f155df 100644 --- a/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/io/zero_copy_stream_unittest.cc @@ -370,6 +370,100 @@ TEST_F(IoTest, GzipIo) { delete [] buffer; } +TEST_F(IoTest, GzipIoWithFlush) { + const int kBufferSize = 2*1024; + uint8* buffer = new uint8[kBufferSize]; + // We start with i = 4 as we want a block size > 6. With block size <= 6 + // Flush() fills up the entire 2K buffer with flush markers and the test + // fails. See documentation for Flush() for more detail. + for (int i = 4; i < kBlockSizeCount; i++) { + for (int j = 0; j < kBlockSizeCount; j++) { + for (int z = 0; z < kBlockSizeCount; z++) { + int gzip_buffer_size = kBlockSizes[z]; + int size; + { + ArrayOutputStream output(buffer, kBufferSize, kBlockSizes[i]); + GzipOutputStream::Options options; + options.format = GzipOutputStream::GZIP; + if (gzip_buffer_size != -1) { + options.buffer_size = gzip_buffer_size; + } + GzipOutputStream gzout(&output, options); + WriteStuff(&gzout); + EXPECT_TRUE(gzout.Flush()); + gzout.Close(); + size = output.ByteCount(); + } + { + ArrayInputStream input(buffer, size, kBlockSizes[j]); + GzipInputStream gzin( + &input, GzipInputStream::GZIP, gzip_buffer_size); + ReadStuff(&gzin); + } + } + } + } + delete [] buffer; +} + +TEST_F(IoTest, GzipIoContiguousFlushes) { + const int kBufferSize = 2*1024; + uint8* buffer = new uint8[kBufferSize]; + + int block_size = kBlockSizes[4]; + int gzip_buffer_size = block_size; + int size; + + ArrayOutputStream output(buffer, kBufferSize, block_size); + GzipOutputStream::Options options; + options.format = GzipOutputStream::GZIP; + if (gzip_buffer_size != -1) { + options.buffer_size = gzip_buffer_size; + } + GzipOutputStream gzout(&output, options); + WriteStuff(&gzout); + EXPECT_TRUE(gzout.Flush()); + EXPECT_TRUE(gzout.Flush()); + gzout.Close(); + size = output.ByteCount(); + + ArrayInputStream input(buffer, size, block_size); + GzipInputStream gzin( + &input, GzipInputStream::GZIP, gzip_buffer_size); + ReadStuff(&gzin); + + delete [] buffer; +} + +TEST_F(IoTest, GzipIoReadAfterFlush) { + const int kBufferSize = 2*1024; + uint8* buffer = new uint8[kBufferSize]; + + int block_size = kBlockSizes[4]; + int gzip_buffer_size = block_size; + int size; + ArrayOutputStream output(buffer, kBufferSize, block_size); + GzipOutputStream::Options options; + options.format = GzipOutputStream::GZIP; + if (gzip_buffer_size != -1) { + options.buffer_size = gzip_buffer_size; + } + + GzipOutputStream gzout(&output, options); + WriteStuff(&gzout); + EXPECT_TRUE(gzout.Flush()); + size = output.ByteCount(); + + ArrayInputStream input(buffer, size, block_size); + GzipInputStream gzin( + &input, GzipInputStream::GZIP, gzip_buffer_size); + ReadStuff(&gzin); + + gzout.Close(); + + delete [] buffer; +} + TEST_F(IoTest, ZlibIo) { const int kBufferSize = 2*1024; uint8* buffer = new uint8[kBufferSize]; diff --git a/third_party/protobuf/src/google/protobuf/lite_unittest.cc b/third_party/protobuf/src/google/protobuf/lite_unittest.cc index ffeec3c..7a0a57b 100644 --- a/third_party/protobuf/src/google/protobuf/lite_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/lite_unittest.cc @@ -33,11 +33,32 @@ #include <string> #include <iostream> -#include <google/protobuf/test_util_lite.h> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/test_util_lite.h> +#include <google/protobuf/unittest_lite.pb.h> using namespace std; +namespace { +// Helper methods to test parsing merge behavior. +void ExpectMessageMerged(const google::protobuf::unittest::TestAllTypesLite& message) { + GOOGLE_CHECK(message.optional_int32() == 3); + GOOGLE_CHECK(message.optional_int64() == 2); + GOOGLE_CHECK(message.optional_string() == "hello"); +} + +void AssignParsingMergeMessages( + google::protobuf::unittest::TestAllTypesLite* msg1, + google::protobuf::unittest::TestAllTypesLite* msg2, + google::protobuf::unittest::TestAllTypesLite* msg3) { + msg1->set_optional_int32(1); + msg2->set_optional_int64(2); + msg3->set_optional_int32(3); + msg3->set_optional_string("hello"); +} + +} // namespace + int main(int argc, char* argv[]) { string data, packed_data; @@ -107,6 +128,58 @@ int main(int argc, char* argv[]) { google::protobuf::TestUtilLite::ExpectPackedExtensionsClear(message); } + { + // Test that if an optional or required message/group field appears multiple + // times in the input, they need to be merged. + google::protobuf::unittest::TestParsingMergeLite::RepeatedFieldsGenerator generator; + google::protobuf::unittest::TestAllTypesLite* msg1; + google::protobuf::unittest::TestAllTypesLite* msg2; + google::protobuf::unittest::TestAllTypesLite* msg3; + +#define ASSIGN_REPEATED_FIELD(FIELD) \ + msg1 = generator.add_##FIELD(); \ + msg2 = generator.add_##FIELD(); \ + msg3 = generator.add_##FIELD(); \ + AssignParsingMergeMessages(msg1, msg2, msg3) + + ASSIGN_REPEATED_FIELD(field1); + ASSIGN_REPEATED_FIELD(field2); + ASSIGN_REPEATED_FIELD(field3); + ASSIGN_REPEATED_FIELD(ext1); + ASSIGN_REPEATED_FIELD(ext2); + +#undef ASSIGN_REPEATED_FIELD +#define ASSIGN_REPEATED_GROUP(FIELD) \ + msg1 = generator.add_##FIELD()->mutable_field1(); \ + msg2 = generator.add_##FIELD()->mutable_field1(); \ + msg3 = generator.add_##FIELD()->mutable_field1(); \ + AssignParsingMergeMessages(msg1, msg2, msg3) + + ASSIGN_REPEATED_GROUP(group1); + ASSIGN_REPEATED_GROUP(group2); + +#undef ASSIGN_REPEATED_GROUP + + string buffer; + generator.SerializeToString(&buffer); + google::protobuf::unittest::TestParsingMergeLite parsing_merge; + parsing_merge.ParseFromString(buffer); + + // Required and optional fields should be merged. + ExpectMessageMerged(parsing_merge.required_all_types()); + ExpectMessageMerged(parsing_merge.optional_all_types()); + ExpectMessageMerged( + parsing_merge.optionalgroup().optional_group_all_types()); + ExpectMessageMerged(parsing_merge.GetExtension( + google::protobuf::unittest::TestParsingMergeLite::optional_ext)); + + // Repeated fields should not be merged. + GOOGLE_CHECK(parsing_merge.repeated_all_types_size() == 3); + GOOGLE_CHECK(parsing_merge.repeatedgroup_size() == 3); + GOOGLE_CHECK(parsing_merge.ExtensionSize( + google::protobuf::unittest::TestParsingMergeLite::repeated_ext) == 3); + } + cout << "PASS" << endl; return 0; } diff --git a/third_party/protobuf/src/google/protobuf/message.cc b/third_party/protobuf/src/google/protobuf/message.cc index 0409a94..ab7efa9 100644 --- a/third_party/protobuf/src/google/protobuf/message.cc +++ b/third_party/protobuf/src/google/protobuf/message.cc @@ -44,11 +44,12 @@ #include <google/protobuf/io/zero_copy_stream_impl.h> #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/generated_message_util.h> #include <google/protobuf/reflection_ops.h> #include <google/protobuf/wire_format.h> #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/stubs/map-util.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -182,9 +183,46 @@ bool Message::SerializePartialToOstream(ostream* output) const { } +// ============================================================================= +// Reflection and associated Template Specializations + Reflection::~Reflection() {} -// =================================================================== +#define HANDLE_TYPE(TYPE, CPPTYPE, CTYPE) \ +template<> \ +const RepeatedField<TYPE>& Reflection::GetRepeatedField<TYPE>( \ + const Message& message, const FieldDescriptor* field) const { \ + return *static_cast<RepeatedField<TYPE>* >( \ + MutableRawRepeatedField(const_cast<Message*>(&message), \ + field, CPPTYPE, CTYPE, NULL)); \ +} \ + \ +template<> \ +RepeatedField<TYPE>* Reflection::MutableRepeatedField<TYPE>( \ + Message* message, const FieldDescriptor* field) const { \ + return static_cast<RepeatedField<TYPE>* >( \ + MutableRawRepeatedField(message, field, CPPTYPE, CTYPE, NULL)); \ +} + +HANDLE_TYPE(int32, FieldDescriptor::CPPTYPE_INT32, -1); +HANDLE_TYPE(int64, FieldDescriptor::CPPTYPE_INT64, -1); +HANDLE_TYPE(uint32, FieldDescriptor::CPPTYPE_UINT32, -1); +HANDLE_TYPE(uint64, FieldDescriptor::CPPTYPE_UINT64, -1); +HANDLE_TYPE(float, FieldDescriptor::CPPTYPE_FLOAT, -1); +HANDLE_TYPE(double, FieldDescriptor::CPPTYPE_DOUBLE, -1); +HANDLE_TYPE(bool, FieldDescriptor::CPPTYPE_BOOL, -1); + + +#undef HANDLE_TYPE + +void* Reflection::MutableRawRepeatedString( + Message* message, const FieldDescriptor* field, bool is_string) const { + return MutableRawRepeatedField(message, field, + FieldDescriptor::CPPTYPE_STRING, FieldOptions::STRING, NULL); +} + + +// ============================================================================= // MessageFactory MessageFactory::~MessageFactory() {} @@ -258,6 +296,7 @@ void GeneratedMessageFactory::RegisterType(const Descriptor* descriptor, } } + const Message* GeneratedMessageFactory::GetPrototype(const Descriptor* type) { { ReaderMutexLock lock(&mutex_); diff --git a/third_party/protobuf/src/google/protobuf/message.h b/third_party/protobuf/src/google/protobuf/message.h index 4bbc641..77faa9a 100644 --- a/third_party/protobuf/src/google/protobuf/message.h +++ b/third_party/protobuf/src/google/protobuf/message.h @@ -79,18 +79,19 @@ // // Same as the last block, but do it dynamically via the Message // // reflection interface. // Message* foo = new Foo; -// Descriptor* descriptor = foo->GetDescriptor(); +// const Descriptor* descriptor = foo->GetDescriptor(); // // // Get the descriptors for the fields we're interested in and verify // // their types. -// FieldDescriptor* text_field = descriptor->FindFieldByName("text"); +// const FieldDescriptor* text_field = descriptor->FindFieldByName("text"); // assert(text_field != NULL); // assert(text_field->type() == FieldDescriptor::TYPE_STRING); -// assert(text_field->label() == FieldDescriptor::TYPE_OPTIONAL); -// FieldDescriptor* numbers_field = descriptor->FindFieldByName("numbers"); +// assert(text_field->label() == FieldDescriptor::LABEL_OPTIONAL); +// const FieldDescriptor* numbers_field = descriptor-> +// FindFieldByName("numbers"); // assert(numbers_field != NULL); // assert(numbers_field->type() == FieldDescriptor::TYPE_INT32); -// assert(numbers_field->label() == FieldDescriptor::TYPE_REPEATED); +// assert(numbers_field->label() == FieldDescriptor::LABEL_REPEATED); // // // Parse the message. // foo->ParseFromString(data); @@ -122,6 +123,7 @@ #include <google/protobuf/message_lite.h> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/descriptor.h> namespace google { @@ -133,17 +135,20 @@ class Reflection; class MessageFactory; // Defined in other files. -class Descriptor; // descriptor.h -class FieldDescriptor; // descriptor.h -class EnumDescriptor; // descriptor.h -class EnumValueDescriptor; // descriptor.h +class UnknownFieldSet; // unknown_field_set.h namespace io { class ZeroCopyInputStream; // zero_copy_stream.h class ZeroCopyOutputStream; // zero_copy_stream.h class CodedInputStream; // coded_stream.h class CodedOutputStream; // coded_stream.h } -class UnknownFieldSet; // unknown_field_set.h + + +template<typename T> +class RepeatedField; // repeated_field.h + +template<typename T> +class RepeatedPtrField; // repeated_field.h // A container to hold message metadata. struct Metadata { @@ -151,12 +156,6 @@ struct Metadata { const Reflection* reflection; }; -// Returns the EnumDescriptor for enum type E, which must be a -// proto-declared enum type. Code generated by the protocol compiler -// will include specializations of this template for each enum type declared. -template <typename E> -const EnumDescriptor* GetEnumDescriptor(); - // Abstract interface for protocol messages. // // See also MessageLite, which contains most every-day operations. Message @@ -360,7 +359,6 @@ class LIBPROTOBUF_EXPORT Message : public MessageLite { // write fields from a Reflection without paying attention to the type. class LIBPROTOBUF_EXPORT Reflection { public: - // TODO(kenton): Remove parameter. inline Reflection() {} virtual ~Reflection(); @@ -390,7 +388,7 @@ class LIBPROTOBUF_EXPORT Reflection { virtual void ClearField(Message* message, const FieldDescriptor* field) const = 0; - // Remove the last element of a repeated field. + // Removes the last element of a repeated field. // We don't provide a way to remove any element other than the last // because it invites inefficient use, such as O(n^2) filtering loops // that should have been O(n). If you want to remove an element other @@ -399,6 +397,10 @@ class LIBPROTOBUF_EXPORT Reflection { // call RemoveLast(). virtual void RemoveLast(Message* message, const FieldDescriptor* field) const = 0; + // Removes the last element of a repeated message field, and returns the + // pointer to the caller. Caller takes ownership of the returned pointer. + virtual Message* ReleaseLast(Message* message, + const FieldDescriptor* field) const = 0; // Swap the complete contents of two messages. virtual void Swap(Message* message1, Message* message2) const = 0; @@ -500,6 +502,16 @@ class LIBPROTOBUF_EXPORT Reflection { virtual Message* MutableMessage(Message* message, const FieldDescriptor* field, MessageFactory* factory = NULL) const = 0; + // Releases the message specified by 'field' and returns the pointer, + // ReleaseMessage() will return the message the message object if it exists. + // Otherwise, it may or may not return NULL. In any case, if the return value + // is non-NULL, the caller takes ownership of the pointer. + // If the field existed (HasField() is true), then the returned pointer will + // be the same as the pointer returned by MutableMessage(). + // This function has the same effect as ClearField(). + virtual Message* ReleaseMessage(Message* message, + const FieldDescriptor* field, + MessageFactory* factory = NULL) const = 0; // Repeated field getters ------------------------------------------ @@ -607,7 +619,39 @@ class LIBPROTOBUF_EXPORT Reflection { MessageFactory* factory = NULL) const = 0; - // Extensions ------------------------------------------------------ + // Repeated field accessors ------------------------------------------------- + // The methods above, e.g. GetRepeatedInt32(msg, fd, index), provide singular + // access to the data in a RepeatedField. The methods below provide aggregate + // access by exposing the RepeatedField object itself with the Message. + // Applying these templates to inappropriate types will lead to an undefined + // reference at link time (e.g. GetRepeatedField<***double>), or possibly a + // template matching error at compile time (e.g. GetRepeatedPtrField<File>). + // + // Usage example: my_doubs = refl->GetRepeatedField<double>(msg, fd); + + // for T = Cord and all protobuf scalar types except enums. + template<typename T> + const RepeatedField<T>& GetRepeatedField( + const Message&, const FieldDescriptor*) const; + + // for T = Cord and all protobuf scalar types except enums. + template<typename T> + RepeatedField<T>* MutableRepeatedField( + Message*, const FieldDescriptor*) const; + + // for T = string, google::protobuf::internal::StringPieceField + // google::protobuf::Message & descendants. + template<typename T> + const RepeatedPtrField<T>& GetRepeatedPtrField( + const Message&, const FieldDescriptor*) const; + + // for T = string, google::protobuf::internal::StringPieceField + // google::protobuf::Message & descendants. + template<typename T> + RepeatedPtrField<T>* MutableRepeatedPtrField( + Message*, const FieldDescriptor*) const; + + // Extensions ---------------------------------------------------------------- // Try to find an extension of this message type by fully-qualified field // name. Returns NULL if no extension is known for this name or number. @@ -619,7 +663,26 @@ class LIBPROTOBUF_EXPORT Reflection { virtual const FieldDescriptor* FindKnownExtensionByNumber( int number) const = 0; + // --------------------------------------------------------------------------- + + protected: + // Obtain a pointer to a Repeated Field Structure and do some type checking: + // on field->cpp_type(), + // on field->field_option().ctype() (if ctype >= 0) + // of field->message_type() (if message_type != NULL). + // We use 1 routine rather than 4 (const vs mutable) x (scalar vs pointer). + virtual void* MutableRawRepeatedField( + Message* message, const FieldDescriptor* field, FieldDescriptor::CppType, + int ctype, const Descriptor* message_type) const = 0; + private: + // Special version for specialized implementations of string. We can't call + // MutableRawRepeatedField directly here because we don't have access to + // FieldOptions::* which are defined in descriptor.pb.h. Including that + // file here is not possible because it would cause a circular include cycle. + void* MutableRawRepeatedString( + Message* message, const FieldDescriptor* field, bool is_string) const; + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(Reflection); }; @@ -682,10 +745,71 @@ class LIBPROTOBUF_EXPORT MessageFactory { static void InternalRegisterGeneratedMessage(const Descriptor* descriptor, const Message* prototype); + private: GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(MessageFactory); }; +// ============================================================================= +// Implementation details for {Get,Mutable}RawRepeatedPtrField. We provide +// specializations for <string>, <StringPieceField> and <Message> and handle +// everything else with the default template which will match any type having +// a method with signature "static const google::protobuf::Descriptor* descriptor()". +// Such a type presumably is a descendant of google::protobuf::Message. + +template<> +inline const RepeatedPtrField<string>& Reflection::GetRepeatedPtrField<string>( + const Message& message, const FieldDescriptor* field) const { + return *static_cast<RepeatedPtrField<string>* >( + MutableRawRepeatedString(const_cast<Message*>(&message), field, true)); +} + +template<> +inline RepeatedPtrField<string>* Reflection::MutableRepeatedPtrField<string>( + Message* message, const FieldDescriptor* field) const { + return static_cast<RepeatedPtrField<string>* >( + MutableRawRepeatedString(message, field, true)); +} + + +// ----- + +template<> +inline const RepeatedPtrField<Message>& Reflection::GetRepeatedPtrField( + const Message& message, const FieldDescriptor* field) const { + return *static_cast<RepeatedPtrField<Message>* >( + MutableRawRepeatedField(const_cast<Message*>(&message), field, + FieldDescriptor::CPPTYPE_MESSAGE, -1, + NULL)); +} + +template<> +inline RepeatedPtrField<Message>* Reflection::MutableRepeatedPtrField( + Message* message, const FieldDescriptor* field) const { + return static_cast<RepeatedPtrField<Message>* >( + MutableRawRepeatedField(message, field, + FieldDescriptor::CPPTYPE_MESSAGE, -1, + NULL)); +} + +template<typename PB> +inline const RepeatedPtrField<PB>& Reflection::GetRepeatedPtrField( + const Message& message, const FieldDescriptor* field) const { + return *static_cast<RepeatedPtrField<PB>* >( + MutableRawRepeatedField(const_cast<Message*>(&message), field, + FieldDescriptor::CPPTYPE_MESSAGE, -1, + PB::default_instance().GetDescriptor())); +} + +template<typename PB> +inline RepeatedPtrField<PB>* Reflection::MutableRepeatedPtrField( + Message* message, const FieldDescriptor* field) const { + return static_cast<RepeatedPtrField<PB>* >( + MutableRawRepeatedField(message, field, + FieldDescriptor::CPPTYPE_MESSAGE, -1, + PB::default_instance().GetDescriptor())); +} + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/message_lite.cc b/third_party/protobuf/src/google/protobuf/message_lite.cc index 7c8f37d..49dbe6e 100644 --- a/third_party/protobuf/src/google/protobuf/message_lite.cc +++ b/third_party/protobuf/src/google/protobuf/message_lite.cc @@ -37,8 +37,8 @@ #include <string> #include <google/protobuf/stubs/common.h> #include <google/protobuf/io/coded_stream.h> -#include <google/protobuf/io/zero_copy_stream_impl.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/io/zero_copy_stream_impl_lite.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { diff --git a/third_party/protobuf/src/google/protobuf/message_lite.h b/third_party/protobuf/src/google/protobuf/message_lite.h index ebf4ba3c..1ec3068 100644 --- a/third_party/protobuf/src/google/protobuf/message_lite.h +++ b/third_party/protobuf/src/google/protobuf/message_lite.h @@ -40,11 +40,17 @@ #define GOOGLE_PROTOBUF_MESSAGE_LITE_H__ #include <google/protobuf/stubs/common.h> -#include <google/protobuf/io/coded_stream.h> namespace google { namespace protobuf { +namespace io { + class CodedInputStream; + class CodedOutputStream; + class ZeroCopyInputStream; + class ZeroCopyOutputStream; +} + // Interface to light weight protocol messages. // // This interface is implemented by all protocol message objects. Non-lite @@ -158,6 +164,7 @@ class LIBPROTOBUF_EXPORT MessageLite { // followed by IsInitialized(). virtual bool MergePartialFromCodedStream(io::CodedInputStream* input) = 0; + // Serialization --------------------------------------------------- // Methods for serializing in protocol buffer format. Most of these // are just simple wrappers around ByteSize() and SerializeWithCachedSizes(). diff --git a/third_party/protobuf/src/google/protobuf/message_unittest.cc b/third_party/protobuf/src/google/protobuf/message_unittest.cc index 33b9e77..8aaeabb 100644 --- a/third_party/protobuf/src/google/protobuf/message_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/message_unittest.cc @@ -257,6 +257,78 @@ TEST(MessageTest, ParseFailsOnInvalidMessageEnd) { EXPECT_FALSE(message.ParseFromArray("\014", 1)); } +namespace { + +void ExpectMessageMerged(const unittest::TestAllTypes& message) { + EXPECT_EQ(3, message.optional_int32()); + EXPECT_EQ(2, message.optional_int64()); + EXPECT_EQ("hello", message.optional_string()); +} + +void AssignParsingMergeMessages( + unittest::TestAllTypes* msg1, + unittest::TestAllTypes* msg2, + unittest::TestAllTypes* msg3) { + msg1->set_optional_int32(1); + msg2->set_optional_int64(2); + msg3->set_optional_int32(3); + msg3->set_optional_string("hello"); +} + +} // namespace + +// Test that if an optional or required message/group field appears multiple +// times in the input, they need to be merged. +TEST(MessageTest, ParsingMerge) { + unittest::TestParsingMerge::RepeatedFieldsGenerator generator; + unittest::TestAllTypes* msg1; + unittest::TestAllTypes* msg2; + unittest::TestAllTypes* msg3; + +#define ASSIGN_REPEATED_FIELD(FIELD) \ + msg1 = generator.add_##FIELD(); \ + msg2 = generator.add_##FIELD(); \ + msg3 = generator.add_##FIELD(); \ + AssignParsingMergeMessages(msg1, msg2, msg3) + + ASSIGN_REPEATED_FIELD(field1); + ASSIGN_REPEATED_FIELD(field2); + ASSIGN_REPEATED_FIELD(field3); + ASSIGN_REPEATED_FIELD(ext1); + ASSIGN_REPEATED_FIELD(ext2); + +#undef ASSIGN_REPEATED_FIELD +#define ASSIGN_REPEATED_GROUP(FIELD) \ + msg1 = generator.add_##FIELD()->mutable_field1(); \ + msg2 = generator.add_##FIELD()->mutable_field1(); \ + msg3 = generator.add_##FIELD()->mutable_field1(); \ + AssignParsingMergeMessages(msg1, msg2, msg3) + + ASSIGN_REPEATED_GROUP(group1); + ASSIGN_REPEATED_GROUP(group2); + +#undef ASSIGN_REPEATED_GROUP + + string buffer; + generator.SerializeToString(&buffer); + unittest::TestParsingMerge parsing_merge; + parsing_merge.ParseFromString(buffer); + + // Required and optional fields should be merged. + ExpectMessageMerged(parsing_merge.required_all_types()); + ExpectMessageMerged(parsing_merge.optional_all_types()); + ExpectMessageMerged( + parsing_merge.optionalgroup().optional_group_all_types()); + ExpectMessageMerged( + parsing_merge.GetExtension(unittest::TestParsingMerge::optional_ext)); + + // Repeated fields should not be merged. + EXPECT_EQ(3, parsing_merge.repeated_all_types_size()); + EXPECT_EQ(3, parsing_merge.repeatedgroup_size()); + EXPECT_EQ(3, parsing_merge.ExtensionSize( + unittest::TestParsingMerge::repeated_ext)); +} + TEST(MessageFactoryTest, GeneratedFactoryLookup) { EXPECT_EQ( MessageFactory::generated_factory()->GetPrototype( @@ -277,5 +349,6 @@ TEST(MessageFactoryTest, GeneratedFactoryUnknownType) { MessageFactory::generated_factory()->GetPrototype(descriptor) == NULL); } + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/reflection_ops.cc b/third_party/protobuf/src/google/protobuf/reflection_ops.cc index 897c0d7..f00997c 100644 --- a/third_party/protobuf/src/google/protobuf/reflection_ops.cc +++ b/third_party/protobuf/src/google/protobuf/reflection_ops.cc @@ -32,8 +32,12 @@ // Based on original Protocol Buffers design by // Sanjay Ghemawat, Jeff Dean, and others. +#include <string> +#include <vector> + #include <google/protobuf/reflection_ops.h> #include <google/protobuf/descriptor.h> +#include <google/protobuf/descriptor.pb.h> #include <google/protobuf/unknown_field_set.h> #include <google/protobuf/stubs/strutil.h> @@ -151,11 +155,12 @@ bool ReflectionOps::IsInitialized(const Message& message) { for (int i = 0; i < fields.size(); i++) { const FieldDescriptor* field = fields[i]; if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { + if (field->is_repeated()) { int size = reflection->FieldSize(message, field); - for (int i = 0; i < size; i++) { - if (!reflection->GetRepeatedMessage(message, field, i) + for (int j = 0; j < size; j++) { + if (!reflection->GetRepeatedMessage(message, field, j) .IsInitialized()) { return false; } @@ -183,8 +188,8 @@ void ReflectionOps::DiscardUnknownFields(Message* message) { if (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) { if (field->is_repeated()) { int size = reflection->FieldSize(*message, field); - for (int i = 0; i < size; i++) { - reflection->MutableRepeatedMessage(message, field, i) + for (int j = 0; j < size; j++) { + reflection->MutableRepeatedMessage(message, field, j) ->DiscardUnknownFields(); } } else { @@ -240,11 +245,11 @@ void ReflectionOps::FindInitializationErrors( if (field->is_repeated()) { int size = reflection->FieldSize(message, field); - for (int i = 0; i < size; i++) { + for (int j = 0; j < size; j++) { const Message& sub_message = - reflection->GetRepeatedMessage(message, field, i); + reflection->GetRepeatedMessage(message, field, j); FindInitializationErrors(sub_message, - SubMessagePrefix(prefix, field, i), + SubMessagePrefix(prefix, field, j), errors); } } else { diff --git a/third_party/protobuf/src/google/protobuf/reflection_ops.h b/third_party/protobuf/src/google/protobuf/reflection_ops.h index 355a0a5..60165c2 100644 --- a/third_party/protobuf/src/google/protobuf/reflection_ops.h +++ b/third_party/protobuf/src/google/protobuf/reflection_ops.h @@ -38,6 +38,7 @@ #ifndef GOOGLE_PROTOBUF_REFLECTION_OPS_H__ #define GOOGLE_PROTOBUF_REFLECTION_OPS_H__ +#include <google/protobuf/stubs/common.h> #include <google/protobuf/message.h> namespace google { diff --git a/third_party/protobuf/src/google/protobuf/repeated_field.cc b/third_party/protobuf/src/google/protobuf/repeated_field.cc index 0937774..2c1f74c 100644 --- a/third_party/protobuf/src/google/protobuf/repeated_field.cc +++ b/third_party/protobuf/src/google/protobuf/repeated_field.cc @@ -46,42 +46,31 @@ void RepeatedPtrFieldBase::Reserve(int new_size) { if (total_size_ >= new_size) return; void** old_elements = elements_; - total_size_ = max(total_size_ * 2, new_size); + total_size_ = max(kMinRepeatedFieldAllocationSize, + max(total_size_ * 2, new_size)); elements_ = new void*[total_size_]; - memcpy(elements_, old_elements, allocated_size_ * sizeof(elements_[0])); - if (old_elements != initial_space_) { + if (old_elements != NULL) { + memcpy(elements_, old_elements, allocated_size_ * sizeof(elements_[0])); delete [] old_elements; } } void RepeatedPtrFieldBase::Swap(RepeatedPtrFieldBase* other) { + if (this == other) return; void** swap_elements = elements_; int swap_current_size = current_size_; int swap_allocated_size = allocated_size_; int swap_total_size = total_size_; - // We may not be using initial_space_ but it's not worth checking. Just - // copy it anyway. - void* swap_initial_space[kInitialSize]; - memcpy(swap_initial_space, initial_space_, sizeof(initial_space_)); elements_ = other->elements_; current_size_ = other->current_size_; allocated_size_ = other->allocated_size_; total_size_ = other->total_size_; - memcpy(initial_space_, other->initial_space_, sizeof(initial_space_)); other->elements_ = swap_elements; other->current_size_ = swap_current_size; other->allocated_size_ = swap_allocated_size; other->total_size_ = swap_total_size; - memcpy(other->initial_space_, swap_initial_space, sizeof(swap_initial_space)); - - if (elements_ == other->initial_space_) { - elements_ = initial_space_; - } - if (other->elements_ == initial_space_) { - other->elements_ = other->initial_space_; - } } string* StringTypeHandlerBase::New() { diff --git a/third_party/protobuf/src/google/protobuf/repeated_field.h b/third_party/protobuf/src/google/protobuf/repeated_field.h index aed4ce9..b9db2b6 100644 --- a/third_party/protobuf/src/google/protobuf/repeated_field.h +++ b/third_party/protobuf/src/google/protobuf/repeated_field.h @@ -46,24 +46,35 @@ #ifndef GOOGLE_PROTOBUF_REPEATED_FIELD_H__ #define GOOGLE_PROTOBUF_REPEATED_FIELD_H__ +#include <algorithm> #include <string> #include <iterator> #include <google/protobuf/stubs/common.h> +#include <google/protobuf/stubs/type_traits.h> +#include <google/protobuf/generated_message_util.h> #include <google/protobuf/message_lite.h> namespace google { +namespace upb { +namespace proto2_bridge_opensource { +class FieldAccessor; +} // namespace protobuf_bridge_opensource +} // namespace upb + namespace protobuf { class Message; namespace internal { -// We need this (from generated_message_reflection.cc). -LIBPROTOBUF_EXPORT int StringSpaceUsedExcludingSelf(const string& str); +static const int kMinRepeatedFieldAllocationSize = 4; +// A utility function for logging that doesn't need any template types. +void LogIndexOutOfBounds(int index, int size); } // namespace internal + // RepeatedField is used to represent repeated fields of a primitive type (in // other words, everything except strings and nested Messages). Most users will // not ever use a RepeatedField directly; they will use the get-by-index, @@ -73,6 +84,8 @@ class RepeatedField { public: RepeatedField(); RepeatedField(const RepeatedField& other); + template <typename Iter> + RepeatedField(Iter begin, const Iter& end); ~RepeatedField(); RepeatedField& operator=(const RepeatedField& other); @@ -85,12 +98,14 @@ class RepeatedField { void Add(const Element& value); Element* Add(); // Remove the last element in the array. - // We don't provide a way to remove any element other than the last - // because it invites inefficient use, such as O(n^2) filtering loops - // that should have been O(n). If you want to remove an element other - // than the last, the best way to do it is to re-arrange the elements - // so that the one you want removed is at the end, then call RemoveLast(). void RemoveLast(); + + // Extract elements with indices in "[start .. start+num-1]". + // Copy them into "elements[0 .. num-1]" if "elements" is not NULL. + // Caution: implementation also moves elements with indices [start+num ..]. + // Calling this routine inside a loop can cause quadratic behavior. + void ExtractSubrange(int start, int num, Element* elements); + void Clear(); void MergeFrom(const RepeatedField& other); void CopyFrom(const RepeatedField& other); @@ -121,25 +136,45 @@ class RepeatedField { typedef Element* iterator; typedef const Element* const_iterator; typedef Element value_type; + typedef value_type& reference; + typedef const value_type& const_reference; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef int size_type; + typedef ptrdiff_t difference_type; iterator begin(); const_iterator begin() const; iterator end(); const_iterator end() const; + // Reverse iterator support + typedef std::reverse_iterator<const_iterator> const_reverse_iterator; + typedef std::reverse_iterator<iterator> reverse_iterator; + reverse_iterator rbegin() { + return reverse_iterator(end()); + } + const_reverse_iterator rbegin() const { + return const_reverse_iterator(end()); + } + reverse_iterator rend() { + return reverse_iterator(begin()); + } + const_reverse_iterator rend() const { + return const_reverse_iterator(begin()); + } + // Returns the number of bytes used by the repeated field, excluding // sizeof(*this) int SpaceUsedExcludingSelf() const; private: - static const int kInitialSize = 4; + static const int kInitialSize = 0; Element* elements_; int current_size_; int total_size_; - Element initial_space_[kInitialSize]; - // Move the contents of |from| into |to|, possibly clobbering |from| in the // process. For primitive types this is just a memcpy(), but it could be // specialized for non-primitive types to, say, swap each element instead. @@ -151,7 +186,21 @@ class RepeatedField { namespace internal { template <typename It> class RepeatedPtrIterator; -template <typename It> class RepeatedPtrOverPtrsIterator; +template <typename It, typename VoidPtr> class RepeatedPtrOverPtrsIterator; +} // namespace internal + +namespace internal { + +// This is a helper template to copy an array of elements effeciently when they +// have a trivial copy constructor, and correctly otherwise. This really +// shouldn't be necessary, but our compiler doesn't optimize std::copy very +// effectively. +template <typename Element, + bool HasTrivialCopy = has_trivial_copy<Element>::value> +struct ElementCopier { + void operator()(Element to[], const Element from[], int array_size); +}; + } // namespace internal namespace internal { @@ -186,6 +235,10 @@ class LIBPROTOBUF_EXPORT RepeatedPtrFieldBase { // use of AddFromCleared(), which is not part of the public interface. friend class ExtensionSet; + // To parse directly into a proto2 generated class, the upb FieldAccessor + // needs to be able to modify a RepeatedPtrFieldBase directly. + friend class LIBPROTOBUF_EXPORT upb::proto2_bridge_opensource::FieldAccessor; + RepeatedPtrFieldBase(); // Must be called from destructor. @@ -209,6 +262,14 @@ class LIBPROTOBUF_EXPORT RepeatedPtrFieldBase { template <typename TypeHandler> void CopyFrom(const RepeatedPtrFieldBase& other); + void CloseGap(int start, int num) { + // Close up a gap of "num" elements starting at offset "start". + for (int i = start + num; i < allocated_size_; ++i) + elements_[i - num] = elements_[i]; + current_size_ -= num; + allocated_size_ -= num; + } + void Reserve(int new_size); int Capacity() const; @@ -250,15 +311,13 @@ class LIBPROTOBUF_EXPORT RepeatedPtrFieldBase { private: GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(RepeatedPtrFieldBase); - static const int kInitialSize = 4; + static const int kInitialSize = 0; void** elements_; int current_size_; int allocated_size_; int total_size_; - void* initial_space_[kInitialSize]; - template <typename TypeHandler> static inline typename TypeHandler::Type* cast(void* element) { return reinterpret_cast<typename TypeHandler::Type*>(element); @@ -280,6 +339,7 @@ class GenericTypeHandler { to->MergeFrom(from); } static int SpaceUsed(const GenericType& value) { return value.SpaceUsed(); } + static const Type& default_instance() { return Type::default_instance(); } }; template <> @@ -288,6 +348,25 @@ inline void GenericTypeHandler<MessageLite>::Merge( to->CheckTypeAndMergeFrom(from); } +template <> +inline const MessageLite& GenericTypeHandler<MessageLite>::default_instance() { + // Yes, the behavior of the code is undefined, but this function is only + // called when we're already deep into the world of undefined, because the + // caller called Get(index) out of bounds. + MessageLite* null = NULL; + return *null; +} + +template <> +inline const Message& GenericTypeHandler<Message>::default_instance() { + // Yes, the behavior of the code is undefined, but this function is only + // called when we're already deep into the world of undefined, because the + // caller called Get(index) out of bounds. + Message* null = NULL; + return *null; +} + + // HACK: If a class is declared as DLL-exported in MSVC, it insists on // generating copies of all its methods -- even inline ones -- to include // in the DLL. But SpaceUsed() calls StringSpaceUsedExcludingSelf() which @@ -303,6 +382,9 @@ class LIBPROTOBUF_EXPORT StringTypeHandlerBase { static void Delete(string* value); static void Clear(string* value) { value->clear(); } static void Merge(const string& from, string* to) { *to = from; } + static const Type& default_instance() { + return ::google::protobuf::internal::kEmptyString; + } }; class StringTypeHandler : public StringTypeHandlerBase { @@ -322,6 +404,8 @@ class RepeatedPtrField : public internal::RepeatedPtrFieldBase { public: RepeatedPtrField(); RepeatedPtrField(const RepeatedPtrField& other); + template <typename Iter> + RepeatedPtrField(Iter begin, const Iter& end); ~RepeatedPtrField(); RepeatedPtrField& operator=(const RepeatedPtrField& other); @@ -331,7 +415,16 @@ class RepeatedPtrField : public internal::RepeatedPtrFieldBase { const Element& Get(int index) const; Element* Mutable(int index); Element* Add(); - void RemoveLast(); // Remove the last element in the array. + + // Remove the last element in the array. + // Ownership of the element is retained by the array. + void RemoveLast(); + + // Delete elements with indices in the range [start .. start+num-1]. + // Caution: implementation moves all elements with indices [start+num .. ]. + // Calling this routine inside a loop can cause quadratic behavior. + void DeleteSubrange(int start, int num); + void Clear(); void MergeFrom(const RepeatedPtrField& other); void CopyFrom(const RepeatedPtrField& other); @@ -358,42 +451,78 @@ class RepeatedPtrField : public internal::RepeatedPtrFieldBase { typedef internal::RepeatedPtrIterator<Element> iterator; typedef internal::RepeatedPtrIterator<const Element> const_iterator; typedef Element value_type; + typedef value_type& reference; + typedef const value_type& const_reference; + typedef value_type* pointer; + typedef const value_type* const_pointer; + typedef int size_type; + typedef ptrdiff_t difference_type; iterator begin(); const_iterator begin() const; iterator end(); const_iterator end() const; + // Reverse iterator support + typedef std::reverse_iterator<const_iterator> const_reverse_iterator; + typedef std::reverse_iterator<iterator> reverse_iterator; + reverse_iterator rbegin() { + return reverse_iterator(end()); + } + const_reverse_iterator rbegin() const { + return const_reverse_iterator(end()); + } + reverse_iterator rend() { + return reverse_iterator(begin()); + } + const_reverse_iterator rend() const { + return const_reverse_iterator(begin()); + } + // Custom STL-like iterator that iterates over and returns the underlying // pointers to Element rather than Element itself. - typedef internal::RepeatedPtrOverPtrsIterator<Element> pointer_iterator; + typedef internal::RepeatedPtrOverPtrsIterator<Element, void*> + pointer_iterator; + typedef internal::RepeatedPtrOverPtrsIterator<const Element, const void*> + const_pointer_iterator; pointer_iterator pointer_begin(); + const_pointer_iterator pointer_begin() const; pointer_iterator pointer_end(); + const_pointer_iterator pointer_end() const; // Returns (an estimate of) the number of bytes used by the repeated field, // excluding sizeof(*this). int SpaceUsedExcludingSelf() const; // Advanced memory management -------------------------------------- - // When hardcore memory management becomes necessary -- as it often + // When hardcore memory management becomes necessary -- as it sometimes // does here at Google -- the following methods may be useful. // Add an already-allocated object, passing ownership to the // RepeatedPtrField. void AddAllocated(Element* value); - // Remove the last element and return it, passing ownership to the - // caller. + // Remove the last element and return it, passing ownership to the caller. // Requires: size() > 0 Element* ReleaseLast(); + // Extract elements with indices in the range "[start .. start+num-1]". + // The caller assumes ownership of the extracted elements and is responsible + // for deleting them when they are no longer needed. + // If "elements" is non-NULL, then pointers to the extracted elements + // are stored in "elements[0 .. num-1]" for the convenience of the caller. + // If "elements" is NULL, then the caller must use some other mechanism + // to perform any further operations (like deletion) on these elements. + // Caution: implementation also moves elements with indices [start+num ..]. + // Calling this routine inside a loop can cause quadratic behavior. + void ExtractSubrange(int start, int num, Element** elements); + // When elements are removed by calls to RemoveLast() or Clear(), they // are not actually freed. Instead, they are cleared and kept so that // they can be reused later. This can save lots of CPU time when // repeatedly reusing a protocol message for similar purposes. // - // Really, extremely hardcore programs may actually want to manipulate - // these objects to better-optimize memory management. These methods - // allow that. + // Hardcore programs may choose to manipulate these cleared objects + // to better optimize memory management using the following routines. // Get the number of cleared objects that are currently being kept // around for reuse. @@ -420,30 +549,40 @@ class RepeatedPtrField : public internal::RepeatedPtrFieldBase { template <typename Element> inline RepeatedField<Element>::RepeatedField() - : elements_(initial_space_), + : elements_(NULL), current_size_(0), total_size_(kInitialSize) { } template <typename Element> inline RepeatedField<Element>::RepeatedField(const RepeatedField& other) - : elements_(initial_space_), + : elements_(NULL), current_size_(0), total_size_(kInitialSize) { CopyFrom(other); } template <typename Element> -RepeatedField<Element>::~RepeatedField() { - if (elements_ != initial_space_) { - delete [] elements_; +template <typename Iter> +inline RepeatedField<Element>::RepeatedField(Iter begin, const Iter& end) + : elements_(NULL), + current_size_(0), + total_size_(kInitialSize) { + for (; begin != end; ++begin) { + Add(*begin); } } template <typename Element> +RepeatedField<Element>::~RepeatedField() { + delete [] elements_; +} + +template <typename Element> inline RepeatedField<Element>& RepeatedField<Element>::operator=(const RepeatedField& other) { - CopyFrom(other); + if (this != &other) + CopyFrom(other); return *this; } @@ -506,15 +645,38 @@ inline void RepeatedField<Element>::RemoveLast() { } template <typename Element> +void RepeatedField<Element>::ExtractSubrange( + int start, int num, Element* elements) { + GOOGLE_DCHECK_GE(start, 0); + GOOGLE_DCHECK_GE(num, 0); + GOOGLE_DCHECK_LE(start + num, this->size()); + + // Save the values of the removed elements if requested. + if (elements != NULL) { + for (int i = 0; i < num; ++i) + elements[i] = this->Get(i + start); + } + + // Slide remaining elements down to fill the gap. + if (num > 0) { + for (int i = start + num; i < this->size(); ++i) + this->Set(i - num, this->Get(i)); + this->Truncate(this->size() - num); + } +} + +template <typename Element> inline void RepeatedField<Element>::Clear() { current_size_ = 0; } template <typename Element> inline void RepeatedField<Element>::MergeFrom(const RepeatedField& other) { - Reserve(current_size_ + other.current_size_); - CopyArray(elements_ + current_size_, other.elements_, other.current_size_); - current_size_ += other.current_size_; + if (other.current_size_ != 0) { + Reserve(current_size_ + other.current_size_); + CopyArray(elements_ + current_size_, other.elements_, other.current_size_); + current_size_ += other.current_size_; + } } template <typename Element> @@ -536,30 +698,18 @@ inline const Element* RepeatedField<Element>::data() const { template <typename Element> void RepeatedField<Element>::Swap(RepeatedField* other) { + if (this == other) return; Element* swap_elements = elements_; int swap_current_size = current_size_; int swap_total_size = total_size_; - // We may not be using initial_space_ but it's not worth checking. Just - // copy it anyway. - Element swap_initial_space[kInitialSize]; - MoveArray(swap_initial_space, initial_space_, kInitialSize); elements_ = other->elements_; current_size_ = other->current_size_; total_size_ = other->total_size_; - MoveArray(initial_space_, other->initial_space_, kInitialSize); other->elements_ = swap_elements; other->current_size_ = swap_current_size; other->total_size_ = swap_total_size; - MoveArray(other->initial_space_, swap_initial_space, kInitialSize); - - if (elements_ == other->initial_space_) { - elements_ = initial_space_; - } - if (other->elements_ == initial_space_) { - other->elements_ = other->initial_space_; - } } template <typename Element> @@ -590,20 +740,21 @@ RepeatedField<Element>::end() const { template <typename Element> inline int RepeatedField<Element>::SpaceUsedExcludingSelf() const { - return (elements_ != initial_space_) ? total_size_ * sizeof(elements_[0]) : 0; + return (elements_ != NULL) ? total_size_ * sizeof(elements_[0]) : 0; } -// Avoid inlining of Reserve(): new, memcpy, and delete[] lead to a significant +// Avoid inlining of Reserve(): new, copy, and delete[] lead to a significant // amount of code bloat. template <typename Element> void RepeatedField<Element>::Reserve(int new_size) { if (total_size_ >= new_size) return; Element* old_elements = elements_; - total_size_ = max(total_size_ * 2, new_size); + total_size_ = max(google::protobuf::internal::kMinRepeatedFieldAllocationSize, + max(total_size_ * 2, new_size)); elements_ = new Element[total_size_]; - MoveArray(elements_, old_elements, current_size_); - if (old_elements != initial_space_) { + if (old_elements != NULL) { + MoveArray(elements_, old_elements, current_size_); delete [] old_elements; } } @@ -617,22 +768,39 @@ inline void RepeatedField<Element>::Truncate(int new_size) { template <typename Element> inline void RepeatedField<Element>::MoveArray( Element to[], Element from[], int array_size) { - memcpy(to, from, array_size * sizeof(Element)); + CopyArray(to, from, array_size); } template <typename Element> inline void RepeatedField<Element>::CopyArray( Element to[], const Element from[], int array_size) { - memcpy(to, from, array_size * sizeof(Element)); + internal::ElementCopier<Element>()(to, from, array_size); } +namespace internal { + +template <typename Element, bool HasTrivialCopy> +void ElementCopier<Element, HasTrivialCopy>::operator()( + Element to[], const Element from[], int array_size) { + std::copy(from, from + array_size, to); +} + +template <typename Element> +struct ElementCopier<Element, true> { + void operator()(Element to[], const Element from[], int array_size) { + memcpy(to, from, array_size * sizeof(Element)); + } +}; + +} // namespace internal + // ------------------------------------------------------------------- namespace internal { inline RepeatedPtrFieldBase::RepeatedPtrFieldBase() - : elements_(initial_space_), + : elements_(NULL), current_size_(0), allocated_size_(0), total_size_(kInitialSize) { @@ -643,16 +811,13 @@ void RepeatedPtrFieldBase::Destroy() { for (int i = 0; i < allocated_size_; i++) { TypeHandler::Delete(cast<TypeHandler>(elements_[i])); } - if (elements_ != initial_space_) { - delete [] elements_; - } + delete [] elements_; } inline int RepeatedPtrFieldBase::size() const { return current_size_; } - template <typename TypeHandler> inline const typename TypeHandler::Type& RepeatedPtrFieldBase::Get(int index) const { @@ -660,6 +825,7 @@ RepeatedPtrFieldBase::Get(int index) const { return *cast<TypeHandler>(elements_[index]); } + template <typename TypeHandler> inline typename TypeHandler::Type* RepeatedPtrFieldBase::Mutable(int index) { @@ -741,7 +907,7 @@ inline void RepeatedPtrFieldBase::SwapElements(int index1, int index2) { template <typename TypeHandler> inline int RepeatedPtrFieldBase::SpaceUsedExcludingSelf() const { int allocated_bytes = - (elements_ != initial_space_) ? total_size_ * sizeof(elements_[0]) : 0; + (elements_ != NULL) ? total_size_ * sizeof(elements_[0]) : 0; for (int i = 0; i < allocated_size_; ++i) { allocated_bytes += TypeHandler::SpaceUsed(*cast<TypeHandler>(elements_[i])); } @@ -798,7 +964,6 @@ inline typename TypeHandler::Type* RepeatedPtrFieldBase::ReleaseLast() { return result; } - inline int RepeatedPtrFieldBase::ClearedCount() const { return allocated_size_ - current_size_; } @@ -822,11 +987,13 @@ inline typename TypeHandler::Type* RepeatedPtrFieldBase::ReleaseCleared() { template <typename Element> class RepeatedPtrField<Element>::TypeHandler - : public internal::GenericTypeHandler<Element> {}; + : public internal::GenericTypeHandler<Element> { +}; template <> class RepeatedPtrField<string>::TypeHandler - : public internal::StringTypeHandler {}; + : public internal::StringTypeHandler { +}; template <typename Element> @@ -839,6 +1006,15 @@ inline RepeatedPtrField<Element>::RepeatedPtrField( } template <typename Element> +template <typename Iter> +inline RepeatedPtrField<Element>::RepeatedPtrField( + Iter begin, const Iter& end) { + for (; begin != end; ++begin) { + *Add() = *begin; + } +} + +template <typename Element> RepeatedPtrField<Element>::~RepeatedPtrField() { Destroy<TypeHandler>(); } @@ -846,7 +1022,8 @@ RepeatedPtrField<Element>::~RepeatedPtrField() { template <typename Element> inline RepeatedPtrField<Element>& RepeatedPtrField<Element>::operator=( const RepeatedPtrField& other) { - CopyFrom(other); + if (this != &other) + CopyFrom(other); return *this; } @@ -860,6 +1037,7 @@ inline const Element& RepeatedPtrField<Element>::Get(int index) const { return RepeatedPtrFieldBase::Get<TypeHandler>(index); } + template <typename Element> inline Element* RepeatedPtrField<Element>::Mutable(int index) { return RepeatedPtrFieldBase::Mutable<TypeHandler>(index); @@ -876,6 +1054,33 @@ inline void RepeatedPtrField<Element>::RemoveLast() { } template <typename Element> +inline void RepeatedPtrField<Element>::DeleteSubrange(int start, int num) { + GOOGLE_DCHECK_GE(start, 0); + GOOGLE_DCHECK_GE(num, 0); + GOOGLE_DCHECK_LE(start + num, size()); + for (int i = 0; i < num; ++i) + delete RepeatedPtrFieldBase::Mutable<TypeHandler>(start + i); + ExtractSubrange(start, num, NULL); +} + +template <typename Element> +inline void RepeatedPtrField<Element>::ExtractSubrange( + int start, int num, Element** elements) { + GOOGLE_DCHECK_GE(start, 0); + GOOGLE_DCHECK_GE(num, 0); + GOOGLE_DCHECK_LE(start + num, size()); + + if (num > 0) { + // Save the values of the removed elements if requested. + if (elements != NULL) { + for (int i = 0; i < num; ++i) + elements[i] = RepeatedPtrFieldBase::Mutable<TypeHandler>(i + start); + } + CloseGap(start, num); + } +} + +template <typename Element> inline void RepeatedPtrField<Element>::Clear() { RepeatedPtrFieldBase::Clear<TypeHandler>(); } @@ -961,7 +1166,7 @@ namespace internal { // refer to this class directly; use RepeatedPtrField<T>::iterator instead. // // The iterator for RepeatedPtrField<T>, RepeatedPtrIterator<T>, is -// very similar to iterator_ptr<T**> in util/gtl/iterator_adaptors-inl.h, +// very similar to iterator_ptr<T**> in util/gtl/iterator_adaptors.h, // but adds random-access operators and is modified to wrap a void** base // iterator (since RepeatedPtrField stores its array as a void* array and // casting void** to T** would violate C++ aliasing rules). @@ -1057,11 +1262,14 @@ class RepeatedPtrIterator // rather than the objects themselves as RepeatedPtrIterator does. // Consider using this when working with stl algorithms that change // the array. -template<typename Element> +// The VoidPtr template parameter holds the type-agnostic pointer value +// referenced by the iterator. It should either be "void *" for a mutable +// iterator, or "const void *" for a constant iterator. +template<typename Element, typename VoidPtr> class RepeatedPtrOverPtrsIterator : public std::iterator<std::random_access_iterator_tag, Element*> { public: - typedef RepeatedPtrOverPtrsIterator<Element> iterator; + typedef RepeatedPtrOverPtrsIterator<Element, VoidPtr> iterator; typedef std::iterator< std::random_access_iterator_tag, Element*> superclass; @@ -1072,7 +1280,7 @@ class RepeatedPtrOverPtrsIterator typedef typename superclass::difference_type difference_type; RepeatedPtrOverPtrsIterator() : it_(NULL) {} - explicit RepeatedPtrOverPtrsIterator(void** it) : it_(it) {} + explicit RepeatedPtrOverPtrsIterator(VoidPtr* it) : it_(it) {} // dereferenceable reference operator*() const { return *reinterpret_cast<Element**>(it_); } @@ -1127,10 +1335,9 @@ class RepeatedPtrOverPtrsIterator friend class RepeatedPtrIterator; // The internal iterator. - void** it_; + VoidPtr* it_; }; - } // namespace internal template <typename Element> @@ -1160,10 +1367,21 @@ RepeatedPtrField<Element>::pointer_begin() { return pointer_iterator(raw_mutable_data()); } template <typename Element> +inline typename RepeatedPtrField<Element>::const_pointer_iterator +RepeatedPtrField<Element>::pointer_begin() const { + return const_pointer_iterator(const_cast<const void**>(raw_mutable_data())); +} +template <typename Element> inline typename RepeatedPtrField<Element>::pointer_iterator RepeatedPtrField<Element>::pointer_end() { return pointer_iterator(raw_mutable_data() + size()); } +template <typename Element> +inline typename RepeatedPtrField<Element>::const_pointer_iterator +RepeatedPtrField<Element>::pointer_end() const { + return const_pointer_iterator( + const_cast<const void**>(raw_mutable_data() + size())); +} // Iterators and helper functions that follow the spirit of the STL @@ -1173,7 +1391,7 @@ RepeatedPtrField<Element>::pointer_end() { // std::copy(some_sequence.begin(), some_sequence.end(), // google::protobuf::RepeatedFieldBackInserter(proto.mutable_sequence())); // -// Ported by johannes from util/gtl/proto-array-iterators-inl.h +// Ported by johannes from util/gtl/proto-array-iterators.h namespace internal { // A back inserter for RepeatedField objects. @@ -1194,7 +1412,7 @@ template<typename T> class RepeatedFieldBackInsertIterator RepeatedFieldBackInsertIterator<T>& operator++() { return *this; } - RepeatedFieldBackInsertIterator<T>& operator++(int ignores_parameter) { + RepeatedFieldBackInsertIterator<T>& operator++(int /* unused */) { return *this; } @@ -1225,7 +1443,7 @@ template<typename T> class RepeatedPtrFieldBackInsertIterator RepeatedPtrFieldBackInsertIterator<T>& operator++() { return *this; } - RepeatedPtrFieldBackInsertIterator<T>& operator++(int ignores_parameter) { + RepeatedPtrFieldBackInsertIterator<T>& operator++(int /* unused */) { return *this; } @@ -1254,7 +1472,7 @@ template<typename T> class AllocatedRepeatedPtrFieldBackInsertIterator return *this; } AllocatedRepeatedPtrFieldBackInsertIterator<T>& operator++( - int ignores_parameter) { + int /* unused */) { return *this; } @@ -1264,16 +1482,22 @@ template<typename T> class AllocatedRepeatedPtrFieldBackInsertIterator } // namespace internal // Provides a back insert iterator for RepeatedField instances, -// similar to std::back_inserter(). Note the identically named -// function for RepeatedPtrField instances. +// similar to std::back_inserter(). template<typename T> internal::RepeatedFieldBackInsertIterator<T> RepeatedFieldBackInserter(RepeatedField<T>* const mutable_field) { return internal::RepeatedFieldBackInsertIterator<T>(mutable_field); } // Provides a back insert iterator for RepeatedPtrField instances, -// similar to std::back_inserter(). Note the identically named -// function for RepeatedField instances. +// similar to std::back_inserter(). +template<typename T> internal::RepeatedPtrFieldBackInsertIterator<T> +RepeatedPtrFieldBackInserter(RepeatedPtrField<T>* const mutable_field) { + return internal::RepeatedPtrFieldBackInsertIterator<T>(mutable_field); +} + +// Special back insert iterator for RepeatedPtrField instances, just in +// case someone wants to write generic template code that can access both +// RepeatedFields and RepeatedPtrFields using a common name. template<typename T> internal::RepeatedPtrFieldBackInsertIterator<T> RepeatedFieldBackInserter(RepeatedPtrField<T>* const mutable_field) { return internal::RepeatedPtrFieldBackInsertIterator<T>(mutable_field); diff --git a/third_party/protobuf/src/google/protobuf/repeated_field_reflection_unittest.cc b/third_party/protobuf/src/google/protobuf/repeated_field_reflection_unittest.cc new file mode 100644 index 0000000..5fcfb00 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/repeated_field_reflection_unittest.cc @@ -0,0 +1,193 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: tgs@google.com (Tom Szymanski) +// +// Test reflection methods for aggregate access to Repeated[Ptr]Fields. +// This test proto2 methods on a proto2 layout. + +#include <google/protobuf/stubs/common.h> +#include <google/protobuf/stubs/stringprintf.h> +#include <google/protobuf/unittest.pb.h> +#include <google/protobuf/test_util.h> +#include <gtest/gtest.h> + +namespace google { +namespace protobuf { + +using unittest::ForeignMessage; +using unittest::TestAllTypes; +using unittest::TestAllExtensions; + +namespace { + +static int Func(int i, int j) { + return i * j; +} + +static string StrFunc(int i, int j) { + string str; + SStringPrintf(&str, "%d", Func(i, 4)); + return str; +} + + +TEST(RepeatedFieldReflectionTest, RegularFields) { + TestAllTypes message; + const Reflection* refl = message.GetReflection(); + const Descriptor* desc = message.GetDescriptor(); + + for (int i = 0; i < 10; ++i) { + message.add_repeated_int32(Func(i, 1)); + message.add_repeated_double(Func(i, 2)); + message.add_repeated_string(StrFunc(i, 5)); + message.add_repeated_foreign_message()->set_c(Func(i, 6)); + } + + // Get FieldDescriptors for all the fields of interest. + const FieldDescriptor* fd_repeated_int32 = + desc->FindFieldByName("repeated_int32"); + const FieldDescriptor* fd_repeated_double = + desc->FindFieldByName("repeated_double"); + const FieldDescriptor* fd_repeated_string = + desc->FindFieldByName("repeated_string"); + const FieldDescriptor* fd_repeated_foreign_message = + desc->FindFieldByName("repeated_foreign_message"); + + // Get RepeatedField objects for all fields of interest. + const RepeatedField<int32>& rf_int32 = + refl->GetRepeatedField<int32>(message, fd_repeated_int32); + const RepeatedField<double>& rf_double = + refl->GetRepeatedField<double>(message, fd_repeated_double); + + // Get mutable RepeatedField objects for all fields of interest. + RepeatedField<int32>* mrf_int32 = + refl->MutableRepeatedField<int32>(&message, fd_repeated_int32); + RepeatedField<double>* mrf_double = + refl->MutableRepeatedField<double>(&message, fd_repeated_double); + + // Get RepeatedPtrField objects for all fields of interest. + const RepeatedPtrField<string>& rpf_string = + refl->GetRepeatedPtrField<string>(message, fd_repeated_string); + const RepeatedPtrField<ForeignMessage>& rpf_foreign_message = + refl->GetRepeatedPtrField<ForeignMessage>( + message, fd_repeated_foreign_message); + const RepeatedPtrField<Message>& rpf_message = + refl->GetRepeatedPtrField<Message>( + message, fd_repeated_foreign_message); + + // Get mutable RepeatedPtrField objects for all fields of interest. + RepeatedPtrField<string>* mrpf_string = + refl->MutableRepeatedPtrField<string>(&message, fd_repeated_string); + RepeatedPtrField<ForeignMessage>* mrpf_foreign_message = + refl->MutableRepeatedPtrField<ForeignMessage>( + &message, fd_repeated_foreign_message); + RepeatedPtrField<Message>* mrpf_message = + refl->MutableRepeatedPtrField<Message>( + &message, fd_repeated_foreign_message); + + // Make sure we can do get and sets through the Repeated[Ptr]Field objects. + for (int i = 0; i < 10; ++i) { + // Check gets through const objects. + EXPECT_EQ(rf_int32.Get(i), Func(i, 1)); + EXPECT_EQ(rf_double.Get(i), Func(i, 2)); + EXPECT_EQ(rpf_string.Get(i), StrFunc(i, 5)); + EXPECT_EQ(rpf_foreign_message.Get(i).c(), Func(i, 6)); + EXPECT_EQ(down_cast<const ForeignMessage&>(rpf_message.Get(i)).c(), + Func(i, 6)); + + // Check gets through mutable objects. + EXPECT_EQ(mrf_int32->Get(i), Func(i, 1)); + EXPECT_EQ(mrf_double->Get(i), Func(i, 2)); + EXPECT_EQ(mrpf_string->Get(i), StrFunc(i, 5)); + EXPECT_EQ(mrpf_foreign_message->Get(i).c(), Func(i, 6)); + EXPECT_EQ(down_cast<const ForeignMessage&>(mrpf_message->Get(i)).c(), + Func(i, 6)); + + // Check sets through mutable objects. + mrf_int32->Set(i, Func(i, -1)); + mrf_double->Set(i, Func(i, -2)); + mrpf_string->Mutable(i)->assign(StrFunc(i, -5)); + mrpf_foreign_message->Mutable(i)->set_c(Func(i, -6)); + EXPECT_EQ(message.repeated_int32(i), Func(i, -1)); + EXPECT_EQ(message.repeated_double(i), Func(i, -2)); + EXPECT_EQ(message.repeated_string(i), StrFunc(i, -5)); + EXPECT_EQ(message.repeated_foreign_message(i).c(), Func(i, -6)); + down_cast<ForeignMessage*>(mrpf_message->Mutable(i))->set_c(Func(i, 7)); + EXPECT_EQ(message.repeated_foreign_message(i).c(), Func(i, 7)); + } + + // Make sure types are checked correctly at runtime. + const FieldDescriptor* fd_optional_int32 = + desc->FindFieldByName("optional_int32"); + EXPECT_DEATH(refl->GetRepeatedField<int32>( + message, fd_optional_int32), "requires a repeated field"); + EXPECT_DEATH(refl->GetRepeatedField<double>( + message, fd_repeated_int32), "not the right type"); + EXPECT_DEATH(refl->GetRepeatedPtrField<TestAllTypes>( + message, fd_repeated_foreign_message), "wrong submessage type"); +} + + + + +TEST(RepeatedFieldReflectionTest, ExtensionFields) { + TestAllExtensions extended_message; + const Reflection* refl = extended_message.GetReflection(); + const Descriptor* desc = extended_message.GetDescriptor(); + + for (int i = 0; i < 10; ++i) { + extended_message.AddExtension( + unittest::repeated_int64_extension, Func(i, 1)); + } + + const FieldDescriptor* fd_repeated_int64_extension = + desc->file()->FindExtensionByName("repeated_int64_extension"); + GOOGLE_CHECK(fd_repeated_int64_extension != NULL); + + const RepeatedField<int64>& rf_int64_extension = + refl->GetRepeatedField<int64>(extended_message, + fd_repeated_int64_extension); + + RepeatedField<int64>* mrf_int64_extension = + refl->MutableRepeatedField<int64>(&extended_message, + fd_repeated_int64_extension); + + for (int i = 0; i < 10; ++i) { + EXPECT_EQ(Func(i, 1), rf_int64_extension.Get(i)); + mrf_int64_extension->Set(i, Func(i, -1)); + EXPECT_EQ(Func(i, -1), + extended_message.GetExtension(unittest::repeated_int64_extension, i)); + } +} + +} // namespace +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/repeated_field_unittest.cc b/third_party/protobuf/src/google/protobuf/repeated_field_unittest.cc index e7e1e99..99d5842 100644 --- a/third_party/protobuf/src/google/protobuf/repeated_field_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/repeated_field_unittest.cc @@ -36,6 +36,7 @@ // other proto2 unittests. #include <algorithm> +#include <limits> #include <list> #include <vector> @@ -46,7 +47,7 @@ #include <google/protobuf/stubs/strutil.h> #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { using protobuf_unittest::TestAllTypes; @@ -54,8 +55,7 @@ using protobuf_unittest::TestAllTypes; namespace protobuf { namespace { -// Test operations on a RepeatedField which is small enough that it does -// not allocate a separate array for storage. +// Test operations on a small RepeatedField. TEST(RepeatedField, Small) { RepeatedField<int> field; @@ -77,7 +77,6 @@ TEST(RepeatedField, Small) { EXPECT_EQ(field.size(), 2); EXPECT_EQ(field.Get(0), 5); EXPECT_EQ(field.Get(1), 23); - EXPECT_EQ(field.SpaceUsedExcludingSelf(), 0); field.RemoveLast(); @@ -87,9 +86,11 @@ TEST(RepeatedField, Small) { field.Clear(); EXPECT_EQ(field.size(), 0); - EXPECT_EQ(field.SpaceUsedExcludingSelf(), 0); + int expected_usage = 4 * sizeof(int); + EXPECT_EQ(field.SpaceUsedExcludingSelf(), expected_usage); } + // Test operations on a RepeatedField which is large enough to allocate a // separate array. TEST(RepeatedField, Large) { @@ -213,10 +214,8 @@ TEST(RepeatedField, ReserveLessThanExisting) { TEST(RepeatedField, MergeFrom) { RepeatedField<int> source, destination; - source.Add(4); source.Add(5); - destination.Add(1); destination.Add(2); destination.Add(3); @@ -224,7 +223,6 @@ TEST(RepeatedField, MergeFrom) { destination.MergeFrom(source); ASSERT_EQ(5, destination.size()); - EXPECT_EQ(1, destination.Get(0)); EXPECT_EQ(2, destination.Get(1)); EXPECT_EQ(3, destination.Get(2)); @@ -234,10 +232,8 @@ TEST(RepeatedField, MergeFrom) { TEST(RepeatedField, CopyFrom) { RepeatedField<int> source, destination; - source.Add(4); source.Add(5); - destination.Add(1); destination.Add(2); destination.Add(3); @@ -245,7 +241,6 @@ TEST(RepeatedField, CopyFrom) { destination.CopyFrom(source); ASSERT_EQ(2, destination.size()); - EXPECT_EQ(4, destination.Get(0)); EXPECT_EQ(5, destination.Get(1)); } @@ -262,12 +257,26 @@ TEST(RepeatedField, CopyConstruct) { EXPECT_EQ(2, destination.Get(1)); } +TEST(RepeatedField, IteratorConstruct) { + vector<int> values; + values.push_back(1); + values.push_back(2); + + RepeatedField<int> field(values.begin(), values.end()); + ASSERT_EQ(values.size(), field.size()); + EXPECT_EQ(values[0], field.Get(0)); + EXPECT_EQ(values[1], field.Get(1)); + + RepeatedField<int> other(field.begin(), field.end()); + ASSERT_EQ(values.size(), other.size()); + EXPECT_EQ(values[0], other.Get(0)); + EXPECT_EQ(values[1], other.Get(1)); +} + TEST(RepeatedField, CopyAssign) { RepeatedField<int> source, destination; - source.Add(4); source.Add(5); - destination.Add(1); destination.Add(2); destination.Add(3); @@ -275,11 +284,24 @@ TEST(RepeatedField, CopyAssign) { destination = source; ASSERT_EQ(2, destination.size()); - EXPECT_EQ(4, destination.Get(0)); EXPECT_EQ(5, destination.Get(1)); } +TEST(RepeatedField, SelfAssign) { + // Verify that assignment to self does not destroy data. + RepeatedField<int> source, *p; + p = &source; + source.Add(7); + source.Add(8); + + *p = source; + + ASSERT_EQ(2, source.size()); + EXPECT_EQ(7, source.Get(0)); + EXPECT_EQ(8, source.Get(1)); +} + TEST(RepeatedField, MutableDataIsMutable) { RepeatedField<int> field; field.Add(1); @@ -315,6 +337,41 @@ TEST(RepeatedField, Truncate) { } +TEST(RepeatedField, ExtractSubrange) { + // Exhaustively test every subrange in arrays of all sizes from 0 through 9. + for (int sz = 0; sz < 10; ++sz) { + for (int num = 0; num <= sz; ++num) { + for (int start = 0; start < sz - num; ++start) { + // Create RepeatedField with sz elements having values 0 through sz-1. + RepeatedField<int32> field; + for (int i = 0; i < sz; ++i) + field.Add(i); + EXPECT_EQ(field.size(), sz); + + // Create a catcher array and call ExtractSubrange. + int32 catcher[10]; + for (int i = 0; i < 10; ++i) + catcher[i] = -1; + field.ExtractSubrange(start, num, catcher); + + // Does the resulting array have the right size? + EXPECT_EQ(field.size(), sz - num); + + // Were the removed elements extracted into the catcher array? + for (int i = 0; i < num; ++i) + EXPECT_EQ(catcher[i], start + i); + EXPECT_EQ(catcher[num], -1); + + // Does the resulting array contain the right values? + for (int i = 0; i < start; ++i) + EXPECT_EQ(field.Get(i), i); + for (int i = start; i < field.size(); ++i) + EXPECT_EQ(field.Get(i), i + num); + } + } + } +} + // =================================================================== // RepeatedPtrField tests. These pretty much just mirror the RepeatedField // tests above. @@ -351,6 +408,7 @@ TEST(RepeatedPtrField, Small) { EXPECT_EQ(field.size(), 0); } + TEST(RepeatedPtrField, Large) { RepeatedPtrField<string> field; @@ -565,10 +623,8 @@ TEST(RepeatedPtrField, AddAlocated) { TEST(RepeatedPtrField, MergeFrom) { RepeatedPtrField<string> source, destination; - source.Add()->assign("4"); source.Add()->assign("5"); - destination.Add()->assign("1"); destination.Add()->assign("2"); destination.Add()->assign("3"); @@ -576,7 +632,6 @@ TEST(RepeatedPtrField, MergeFrom) { destination.MergeFrom(source); ASSERT_EQ(5, destination.size()); - EXPECT_EQ("1", destination.Get(0)); EXPECT_EQ("2", destination.Get(1)); EXPECT_EQ("3", destination.Get(2)); @@ -586,10 +641,8 @@ TEST(RepeatedPtrField, MergeFrom) { TEST(RepeatedPtrField, CopyFrom) { RepeatedPtrField<string> source, destination; - source.Add()->assign("4"); source.Add()->assign("5"); - destination.Add()->assign("1"); destination.Add()->assign("2"); destination.Add()->assign("3"); @@ -597,14 +650,12 @@ TEST(RepeatedPtrField, CopyFrom) { destination.CopyFrom(source); ASSERT_EQ(2, destination.size()); - EXPECT_EQ("4", destination.Get(0)); EXPECT_EQ("5", destination.Get(1)); } TEST(RepeatedPtrField, CopyConstruct) { RepeatedPtrField<string> source; - source.Add()->assign("1"); source.Add()->assign("2"); @@ -615,12 +666,45 @@ TEST(RepeatedPtrField, CopyConstruct) { EXPECT_EQ("2", destination.Get(1)); } +TEST(RepeatedPtrField, IteratorConstruct_String) { + vector<string> values; + values.push_back("1"); + values.push_back("2"); + + RepeatedPtrField<string> field(values.begin(), values.end()); + ASSERT_EQ(values.size(), field.size()); + EXPECT_EQ(values[0], field.Get(0)); + EXPECT_EQ(values[1], field.Get(1)); + + RepeatedPtrField<string> other(field.begin(), field.end()); + ASSERT_EQ(values.size(), other.size()); + EXPECT_EQ(values[0], other.Get(0)); + EXPECT_EQ(values[1], other.Get(1)); +} + +TEST(RepeatedPtrField, IteratorConstruct_Proto) { + typedef TestAllTypes::NestedMessage Nested; + vector<Nested> values; + values.push_back(Nested()); + values.back().set_bb(1); + values.push_back(Nested()); + values.back().set_bb(2); + + RepeatedPtrField<Nested> field(values.begin(), values.end()); + ASSERT_EQ(values.size(), field.size()); + EXPECT_EQ(values[0].bb(), field.Get(0).bb()); + EXPECT_EQ(values[1].bb(), field.Get(1).bb()); + + RepeatedPtrField<Nested> other(field.begin(), field.end()); + ASSERT_EQ(values.size(), other.size()); + EXPECT_EQ(values[0].bb(), other.Get(0).bb()); + EXPECT_EQ(values[1].bb(), other.Get(1).bb()); +} + TEST(RepeatedPtrField, CopyAssign) { RepeatedPtrField<string> source, destination; - source.Add()->assign("4"); source.Add()->assign("5"); - destination.Add()->assign("1"); destination.Add()->assign("2"); destination.Add()->assign("3"); @@ -628,11 +712,24 @@ TEST(RepeatedPtrField, CopyAssign) { destination = source; ASSERT_EQ(2, destination.size()); - EXPECT_EQ("4", destination.Get(0)); EXPECT_EQ("5", destination.Get(1)); } +TEST(RepeatedPtrField, SelfAssign) { + // Verify that assignment to self does not destroy data. + RepeatedPtrField<string> source, *p; + p = &source; + source.Add()->assign("7"); + source.Add()->assign("8"); + + *p = source; + + ASSERT_EQ(2, source.size()); + EXPECT_EQ("7", source.Get(0)); + EXPECT_EQ("8", source.Get(1)); +} + TEST(RepeatedPtrField, MutableDataIsMutable) { RepeatedPtrField<string> field; *field.Add() = "1"; @@ -644,6 +741,77 @@ TEST(RepeatedPtrField, MutableDataIsMutable) { EXPECT_EQ("2", field.Get(0)); } +TEST(RepeatedPtrField, ExtractSubrange) { + // Exhaustively test every subrange in arrays of all sizes from 0 through 9 + // with 0 through 3 cleared elements at the end. + for (int sz = 0; sz < 10; ++sz) { + for (int num = 0; num <= sz; ++num) { + for (int start = 0; start < sz - num; ++start) { + for (int extra = 0; extra < 4; ++extra) { + vector<string*> subject; + + // Create an array with "sz" elements and "extra" cleared elements. + RepeatedPtrField<string> field; + for (int i = 0; i < sz + extra; ++i) { + subject.push_back(new string()); + field.AddAllocated(subject[i]); + } + EXPECT_EQ(field.size(), sz + extra); + for (int i = 0; i < extra; ++i) + field.RemoveLast(); + EXPECT_EQ(field.size(), sz); + EXPECT_EQ(field.ClearedCount(), extra); + + // Create a catcher array and call ExtractSubrange. + string* catcher[10]; + for (int i = 0; i < 10; ++i) + catcher[i] = NULL; + field.ExtractSubrange(start, num, catcher); + + // Does the resulting array have the right size? + EXPECT_EQ(field.size(), sz - num); + + // Were the removed elements extracted into the catcher array? + for (int i = 0; i < num; ++i) + EXPECT_EQ(catcher[i], subject[start + i]); + EXPECT_EQ(NULL, catcher[num]); + + // Does the resulting array contain the right values? + for (int i = 0; i < start; ++i) + EXPECT_EQ(field.Mutable(i), subject[i]); + for (int i = start; i < field.size(); ++i) + EXPECT_EQ(field.Mutable(i), subject[i + num]); + + // Reinstate the cleared elements. + EXPECT_EQ(field.ClearedCount(), extra); + for (int i = 0; i < extra; ++i) + field.Add(); + EXPECT_EQ(field.ClearedCount(), 0); + EXPECT_EQ(field.size(), sz - num + extra); + + // Make sure the extra elements are all there (in some order). + for (int i = sz; i < sz + extra; ++i) { + int count = 0; + for (int j = sz; j < sz + extra; ++j) { + if (field.Mutable(j - num) == subject[i]) + count += 1; + } + EXPECT_EQ(count, 1); + } + + // Release the caught elements. + for (int i = 0; i < num; ++i) + delete catcher[i]; + } + } + } + } +} + +TEST(RepeatedPtrField, DeleteSubrange) { + // DeleteSubrange is a trivial extension of ExtendSubrange. +} + // =================================================================== // Iterator tests stolen from net/proto/proto-array_unittest. @@ -738,6 +906,30 @@ TEST_F(RepeatedPtrFieldIteratorTest, ConstIteration) { EXPECT_EQ("baz", *(--const_proto_array.end())); } +TEST_F(RepeatedPtrFieldIteratorTest, MutableReverseIteration) { + RepeatedPtrField<string>::reverse_iterator iter = proto_array_.rbegin(); + EXPECT_EQ("baz", *iter); + ++iter; + EXPECT_EQ("bar", *(iter++)); + EXPECT_EQ("foo", *iter); + ++iter; + EXPECT_TRUE(proto_array_.rend() == iter); + EXPECT_EQ("foo", *(--proto_array_.rend())); +} + +TEST_F(RepeatedPtrFieldIteratorTest, ConstReverseIteration) { + const RepeatedPtrField<string>& const_proto_array = proto_array_; + RepeatedPtrField<string>::const_reverse_iterator iter + = const_proto_array.rbegin(); + EXPECT_EQ("baz", *iter); + ++iter; + EXPECT_EQ("bar", *(iter++)); + EXPECT_EQ("foo", *iter); + ++iter; + EXPECT_TRUE(const_proto_array.rend() == iter); + EXPECT_EQ("foo", *(--const_proto_array.rend())); +} + TEST_F(RepeatedPtrFieldIteratorTest, RandomAccess) { RepeatedPtrField<string>::iterator iter = proto_array_.begin(); RepeatedPtrField<string>::iterator iter2 = iter; @@ -805,9 +997,11 @@ class RepeatedPtrFieldPtrsIteratorTest : public testing::Test { proto_array_.Add()->assign("foo"); proto_array_.Add()->assign("bar"); proto_array_.Add()->assign("baz"); + const_proto_array_ = &proto_array_; } RepeatedPtrField<string> proto_array_; + const RepeatedPtrField<string>* const_proto_array_; }; TEST_F(RepeatedPtrFieldPtrsIteratorTest, ConvertiblePtr) { @@ -815,6 +1009,11 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, ConvertiblePtr) { proto_array_.pointer_begin(); } +TEST_F(RepeatedPtrFieldPtrsIteratorTest, ConvertibleConstPtr) { + RepeatedPtrField<string>::const_pointer_iterator iter = + const_proto_array_->pointer_begin(); +} + TEST_F(RepeatedPtrFieldPtrsIteratorTest, MutablePtrIteration) { RepeatedPtrField<string>::pointer_iterator iter = proto_array_.pointer_begin(); @@ -827,6 +1026,18 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, MutablePtrIteration) { EXPECT_EQ("baz", **(--proto_array_.pointer_end())); } +TEST_F(RepeatedPtrFieldPtrsIteratorTest, MutableConstPtrIteration) { + RepeatedPtrField<string>::const_pointer_iterator iter = + const_proto_array_->pointer_begin(); + EXPECT_EQ("foo", **iter); + ++iter; + EXPECT_EQ("bar", **(iter++)); + EXPECT_EQ("baz", **iter); + ++iter; + EXPECT_TRUE(const_proto_array_->pointer_end() == iter); + EXPECT_EQ("baz", **(--const_proto_array_->pointer_end())); +} + TEST_F(RepeatedPtrFieldPtrsIteratorTest, RandomPtrAccess) { RepeatedPtrField<string>::pointer_iterator iter = proto_array_.pointer_begin(); @@ -840,6 +1051,19 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, RandomPtrAccess) { EXPECT_EQ(3, proto_array_.end() - proto_array_.begin()); } +TEST_F(RepeatedPtrFieldPtrsIteratorTest, RandomConstPtrAccess) { + RepeatedPtrField<string>::const_pointer_iterator iter = + const_proto_array_->pointer_begin(); + RepeatedPtrField<string>::const_pointer_iterator iter2 = iter; + ++iter2; + ++iter2; + EXPECT_TRUE(iter + 2 == iter2); + EXPECT_TRUE(iter == iter2 - 2); + EXPECT_EQ("baz", *iter[2]); + EXPECT_EQ("baz", **(iter + 2)); + EXPECT_EQ(3, const_proto_array_->end() - const_proto_array_->begin()); +} + TEST_F(RepeatedPtrFieldPtrsIteratorTest, ComparablePtr) { RepeatedPtrField<string>::pointer_iterator iter = proto_array_.pointer_begin(); @@ -854,6 +1078,20 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, ComparablePtr) { EXPECT_TRUE(iter >= iter); } +TEST_F(RepeatedPtrFieldPtrsIteratorTest, ComparableConstPtr) { + RepeatedPtrField<string>::const_pointer_iterator iter = + const_proto_array_->pointer_begin(); + RepeatedPtrField<string>::const_pointer_iterator iter2 = iter + 1; + EXPECT_TRUE(iter == iter); + EXPECT_TRUE(iter != iter2); + EXPECT_TRUE(iter < iter2); + EXPECT_TRUE(iter <= iter2); + EXPECT_TRUE(iter <= iter); + EXPECT_TRUE(iter2 > iter); + EXPECT_TRUE(iter2 >= iter); + EXPECT_TRUE(iter >= iter); +} + // Uninitialized iterator does not point to any of the RepeatedPtrOverPtrs. // Dereferencing an uninitialized iterator crashes the process. TEST_F(RepeatedPtrFieldPtrsIteratorTest, UninitializedPtrIterator) { @@ -865,6 +1103,14 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, UninitializedPtrIterator) { EXPECT_TRUE(iter != proto_array_.pointer_end()); } +TEST_F(RepeatedPtrFieldPtrsIteratorTest, UninitializedConstPtrIterator) { + RepeatedPtrField<string>::const_pointer_iterator iter; + EXPECT_TRUE(iter != const_proto_array_->pointer_begin()); + EXPECT_TRUE(iter != const_proto_array_->pointer_begin() + 1); + EXPECT_TRUE(iter != const_proto_array_->pointer_begin() + 2); + EXPECT_TRUE(iter != const_proto_array_->pointer_begin() + 3); + EXPECT_TRUE(iter != const_proto_array_->pointer_end()); +} // This comparison functor is required by the tests for RepeatedPtrOverPtrs. // They operate on strings and need to compare strings as strings in @@ -889,17 +1135,33 @@ TEST_F(RepeatedPtrFieldPtrsIteratorTest, PtrSTLAlgorithms_lower_bound) { proto_array_.Add()->assign("x"); proto_array_.Add()->assign("y"); - RepeatedPtrField<string>::pointer_iterator iter = - proto_array_.pointer_begin(); - string v = "f"; - RepeatedPtrField<string>::pointer_iterator it = - lower_bound(proto_array_.pointer_begin(), proto_array_.pointer_end(), - &v, StringLessThan()); + { + RepeatedPtrField<string>::pointer_iterator iter = + proto_array_.pointer_begin(); + string v = "f"; + RepeatedPtrField<string>::pointer_iterator it = + lower_bound(proto_array_.pointer_begin(), proto_array_.pointer_end(), + &v, StringLessThan()); - GOOGLE_CHECK(*it != NULL); + GOOGLE_CHECK(*it != NULL); - EXPECT_EQ(**it, "n"); - EXPECT_TRUE(it == proto_array_.pointer_begin() + 3); + EXPECT_EQ(**it, "n"); + EXPECT_TRUE(it == proto_array_.pointer_begin() + 3); + } + { + RepeatedPtrField<string>::const_pointer_iterator iter = + const_proto_array_->pointer_begin(); + string v = "f"; + RepeatedPtrField<string>::const_pointer_iterator it = + lower_bound(const_proto_array_->pointer_begin(), + const_proto_array_->pointer_end(), + &v, StringLessThan()); + + GOOGLE_CHECK(*it != NULL); + + EXPECT_EQ(**it, "n"); + EXPECT_TRUE(it == const_proto_array_->pointer_begin() + 3); + } } TEST_F(RepeatedPtrFieldPtrsIteratorTest, PtrMutation) { @@ -1025,13 +1287,24 @@ TEST_F(RepeatedFieldInsertionIteratorsTest, Halves) { TEST_F(RepeatedFieldInsertionIteratorsTest, Words) { ASSERT_EQ(words.size(), protobuffer.repeated_string_size()); - EXPECT_EQ(words.at(0), protobuffer.repeated_string(0)); - EXPECT_EQ(words.at(1), protobuffer.repeated_string(1)); - EXPECT_EQ(words.at(2), protobuffer.repeated_string(2)); - EXPECT_EQ(words.at(3), protobuffer.repeated_string(3)); - EXPECT_EQ(words.at(4), protobuffer.repeated_string(4)); - EXPECT_EQ(words.at(5), protobuffer.repeated_string(5)); - EXPECT_EQ(words.at(6), protobuffer.repeated_string(6)); + for (int i = 0; i < words.size(); ++i) + EXPECT_EQ(words.at(i), protobuffer.repeated_string(i)); +} + +TEST_F(RepeatedFieldInsertionIteratorsTest, Words2) { + words.clear(); + words.push_back("sing"); + words.push_back("a"); + words.push_back("song"); + words.push_back("of"); + words.push_back("six"); + words.push_back("pence"); + protobuffer.mutable_repeated_string()->Clear(); + std::copy(words.begin(), words.end(), RepeatedPtrFieldBackInserter( + protobuffer.mutable_repeated_string())); + ASSERT_EQ(words.size(), protobuffer.repeated_string_size()); + for (int i = 0; i < words.size(); ++i) + EXPECT_EQ(words.at(i), protobuffer.repeated_string(i)); } TEST_F(RepeatedFieldInsertionIteratorsTest, Nesteds) { diff --git a/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_atomicword_compat.h b/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_atomicword_compat.h index e9d8679..8d9a726 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_atomicword_compat.h +++ b/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_atomicword_compat.h @@ -44,7 +44,7 @@ // On LP64 platforms, AtomicWord and Atomic64 are both always long, // so this problem doesn't occur. -#if !defined(GOOGLE_PROTOBUF_ARCH_64_BIT) +#if !defined(GOOGLE_PROTOBUF_HOST_ARCH_64_BIT) namespace google { namespace protobuf { @@ -117,6 +117,6 @@ inline AtomicWord Release_Load(volatile const AtomicWord* ptr) { } // namespace protobuf } // namespace google -#endif // !defined(GOOGLE_PROTOBUF_ARCH_64_BIT) +#endif // !defined(GOOGLE_PROTOBUF_HOST_ARCH_64_BIT) #endif // GOOGLE_PROTOBUF_ATOMICOPS_INTERNALS_ATOMICWORD_COMPAT_H_ diff --git a/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_macosx.h b/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_macosx.h index f9b7581..9af6d636 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_macosx.h +++ b/third_party/protobuf/src/google/protobuf/stubs/atomicops_internals_macosx.h @@ -136,7 +136,7 @@ inline Atomic64 NoBarrier_CompareAndSwap(volatile Atomic64* ptr, Atomic64 prev_value; do { if (OSAtomicCompareAndSwap64(old_value, new_value, - reinterpret_cast<volatile int64_t*>(ptr))) { + const_cast<Atomic64*>(ptr))) { return old_value; } prev_value = *ptr; @@ -150,19 +150,18 @@ inline Atomic64 NoBarrier_AtomicExchange(volatile Atomic64* ptr, do { old_value = *ptr; } while (!OSAtomicCompareAndSwap64(old_value, new_value, - reinterpret_cast<volatile int64_t*>(ptr))); + const_cast<Atomic64*>(ptr))); return old_value; } inline Atomic64 NoBarrier_AtomicIncrement(volatile Atomic64* ptr, Atomic64 increment) { - return OSAtomicAdd64(increment, reinterpret_cast<volatile int64_t*>(ptr)); + return OSAtomicAdd64(increment, const_cast<Atomic64*>(ptr)); } inline Atomic64 Barrier_AtomicIncrement(volatile Atomic64* ptr, Atomic64 increment) { - return OSAtomicAdd64Barrier(increment, - reinterpret_cast<volatile int64_t*>(ptr)); + return OSAtomicAdd64Barrier(increment, const_cast<Atomic64*>(ptr)); } inline Atomic64 Acquire_CompareAndSwap(volatile Atomic64* ptr, @@ -170,8 +169,8 @@ inline Atomic64 Acquire_CompareAndSwap(volatile Atomic64* ptr, Atomic64 new_value) { Atomic64 prev_value; do { - if (OSAtomicCompareAndSwap64Barrier( - old_value, new_value, reinterpret_cast<volatile int64_t*>(ptr))) { + if (OSAtomicCompareAndSwap64Barrier(old_value, new_value, + const_cast<Atomic64*>(ptr))) { return old_value; } prev_value = *ptr; diff --git a/third_party/protobuf/src/google/protobuf/stubs/common.cc b/third_party/protobuf/src/google/protobuf/stubs/common.cc index 7b15be4..16d7852 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/common.cc +++ b/third_party/protobuf/src/google/protobuf/stubs/common.cc @@ -320,6 +320,24 @@ void Mutex::AssertHeld() { #endif // =================================================================== +// emulates google3/util/endian/endian.h +// +// TODO(xiaofeng): PROTOBUF_LITTLE_ENDIAN is unfortunately defined in +// google/protobuf/io/coded_stream.h and therefore can not be used here. +// Maybe move that macro definition here in the furture. +uint32 ghtonl(uint32 x) { + union { + uint32 result; + uint8 result_array[4]; + }; + result_array[0] = static_cast<uint8>(x >> 24); + result_array[1] = static_cast<uint8>((x >> 16) & 0xFF); + result_array[2] = static_cast<uint8>((x >> 8) & 0xFF); + result_array[3] = static_cast<uint8>(x & 0xFF); + return result; +} + +// =================================================================== // Shutdown support. namespace internal { diff --git a/third_party/protobuf/src/google/protobuf/stubs/common.h b/third_party/protobuf/src/google/protobuf/stubs/common.h index cad22fa..4c3e0cb 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/common.h +++ b/third_party/protobuf/src/google/protobuf/stubs/common.h @@ -356,6 +356,12 @@ inline To down_cast(From* f) { // so we only accept pointers return static_cast<To>(f); } +// Simplified down_cast for reference type. +template<typename To, typename From> +inline To down_cast(From& f) { + return static_cast<To>(f); +} + } // namespace internal // We made these internal so that they would show up as such in the docs, @@ -701,6 +707,7 @@ class LIBPROTOBUF_EXPORT LogFinisher { #undef GOOGLE_CHECK_LE #undef GOOGLE_CHECK_GT #undef GOOGLE_CHECK_GE +#undef GOOGLE_CHECK_NOTNULL #undef GOOGLE_DLOG #undef GOOGLE_DCHECK @@ -727,6 +734,18 @@ class LIBPROTOBUF_EXPORT LogFinisher { #define GOOGLE_CHECK_GT(A, B) GOOGLE_CHECK((A) > (B)) #define GOOGLE_CHECK_GE(A, B) GOOGLE_CHECK((A) >= (B)) +namespace internal { +template<typename T> +T* CheckNotNull(const char *file, int line, const char *name, T* val) { + if (val == NULL) { + GOOGLE_LOG(FATAL) << name; + } + return val; +} +} // namespace internal +#define GOOGLE_CHECK_NOTNULL(A) \ + internal::CheckNotNull(__FILE__, __LINE__, "'" #A "' must not be NULL", (A)) + #ifdef NDEBUG #define GOOGLE_DLOG GOOGLE_LOG_IF(INFO, false) @@ -1151,20 +1170,10 @@ using internal::WriterMutexLock; using internal::MutexLockMaybe; // =================================================================== -// from google3/base/type_traits.h +// from google3/util/utf8/public/unilib.h namespace internal { -// Specified by TR1 [4.7.4] Pointer modifications. -template<typename T> struct remove_pointer { typedef T type; }; -template<typename T> struct remove_pointer<T*> { typedef T type; }; -template<typename T> struct remove_pointer<T* const> { typedef T type; }; -template<typename T> struct remove_pointer<T* volatile> { typedef T type; }; -template<typename T> struct remove_pointer<T* const volatile> { - typedef T type; }; - -// =================================================================== - // Checks if the buffer contains structurally-valid UTF-8. Implemented in // structurally_valid.cc. LIBPROTOBUF_EXPORT bool IsStructurallyValidUTF8(const char* buf, int len); @@ -1172,6 +1181,10 @@ LIBPROTOBUF_EXPORT bool IsStructurallyValidUTF8(const char* buf, int len); } // namespace internal // =================================================================== +// from google3/util/endian/endian.h +LIBPROTOBUF_EXPORT uint32 ghtonl(uint32 x); + +// =================================================================== // Shutdown support. // Shut down the entire protocol buffers library, deleting all static-duration diff --git a/third_party/protobuf/src/google/protobuf/stubs/hash.h b/third_party/protobuf/src/google/protobuf/stubs/hash.h index 822d605..f7d1071 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/hash.h +++ b/third_party/protobuf/src/google/protobuf/stubs/hash.h @@ -90,12 +90,16 @@ template <typename Key, typename Data, typename HashFcn = hash<Key>, typename EqualKey = int > class hash_map : public std::map<Key, Data, HashFcn> { + public: + hash_map(int = 0) {} }; template <typename Key, typename HashFcn = hash<Key>, typename EqualKey = int > class hash_set : public std::set<Key, HashFcn> { + public: + hash_set(int = 0) {} }; #elif defined(_MSC_VER) && !defined(_STLPORT_VERSION) @@ -123,6 +127,8 @@ template <typename Key, typename Data, typename EqualKey = int > class hash_map : public HASH_NAMESPACE::hash_map< Key, Data, HashFcn> { + public: + hash_map(int = 0) {} }; template <typename Key, @@ -130,6 +136,8 @@ template <typename Key, typename EqualKey = int > class hash_set : public HASH_NAMESPACE::hash_set< Key, HashFcn> { + public: + hash_set(int = 0) {} }; #else @@ -163,6 +171,8 @@ template <typename Key, typename Data, typename EqualKey = std::equal_to<Key> > class hash_map : public HASH_NAMESPACE::HASH_MAP_CLASS< Key, Data, HashFcn, EqualKey> { + public: + hash_map(int = 0) {} }; template <typename Key, @@ -170,6 +180,8 @@ template <typename Key, typename EqualKey = std::equal_to<Key> > class hash_set : public HASH_NAMESPACE::HASH_SET_CLASS< Key, HashFcn, EqualKey> { + public: + hash_set(int = 0) {} }; #endif diff --git a/third_party/protobuf/src/google/protobuf/stubs/map-util.h b/third_party/protobuf/src/google/protobuf/stubs/map-util.h index f5c9d6b..775848b 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/map-util.h +++ b/third_party/protobuf/src/google/protobuf/stubs/map-util.h @@ -68,6 +68,30 @@ FindOrNull(const Collection& collection, return &it->second; } +// Perform a lookup in a map or hash_map, assuming that the key exists. +// Crash if it does not. +// +// This is intended as a replacement for operator[] as an rvalue (for reading) +// when the key is guaranteed to exist. +// +// operator[] is discouraged for several reasons: +// * It has a side-effect of inserting missing keys +// * It is not thread-safe (even when it is not inserting, it can still +// choose to resize the underlying storage) +// * It invalidates iterators (when it chooses to resize) +// * It default constructs a value object even if it doesn't need to +// +// This version assumes the key is printable, and includes it in the fatal log +// message. +template <class Collection> +const typename Collection::value_type::second_type& +FindOrDie(const Collection& collection, + const typename Collection::value_type::first_type& key) { + typename Collection::const_iterator it = collection.find(key); + GOOGLE_CHECK(it != collection.end()) << "Map key not found: " << key; + return it->second; +} + // Perform a lookup in a map or hash_map whose values are pointers. // If the key is present a const pointer to the associated value is returned, // otherwise a NULL pointer is returned. diff --git a/third_party/protobuf/src/google/protobuf/stubs/stl_util.h b/third_party/protobuf/src/google/protobuf/stubs/stl_util.h new file mode 100644 index 0000000..9021dad --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/stl_util.h @@ -0,0 +1,121 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// from google3/util/gtl/stl_util.h + +#ifndef GOOGLE_PROTOBUF_STUBS_STL_UTIL_H__ +#define GOOGLE_PROTOBUF_STUBS_STL_UTIL_H__ + +#include <google/protobuf/stubs/common.h> + +namespace google { +namespace protobuf { + +// STLDeleteContainerPointers() +// For a range within a container of pointers, calls delete +// (non-array version) on these pointers. +// NOTE: for these three functions, we could just implement a DeleteObject +// functor and then call for_each() on the range and functor, but this +// requires us to pull in all of algorithm.h, which seems expensive. +// For hash_[multi]set, it is important that this deletes behind the iterator +// because the hash_set may call the hash function on the iterator when it is +// advanced, which could result in the hash function trying to deference a +// stale pointer. +template <class ForwardIterator> +void STLDeleteContainerPointers(ForwardIterator begin, + ForwardIterator end) { + while (begin != end) { + ForwardIterator temp = begin; + ++begin; + delete *temp; + } +} + +// Inside Google, this function implements a horrible, disgusting hack in which +// we reach into the string's private implementation and resize it without +// initializing the new bytes. In some cases doing this can significantly +// improve performance. However, since it's totally non-portable it has no +// place in open source code. Feel free to fill this function in with your +// own disgusting hack if you want the perf boost. +inline void STLStringResizeUninitialized(string* s, size_t new_size) { + s->resize(new_size); +} + +// Return a mutable char* pointing to a string's internal buffer, +// which may not be null-terminated. Writing through this pointer will +// modify the string. +// +// string_as_array(&str)[i] is valid for 0 <= i < str.size() until the +// next call to a string method that invalidates iterators. +// +// As of 2006-04, there is no standard-blessed way of getting a +// mutable reference to a string's internal buffer. However, issue 530 +// (http://www.open-std.org/JTC1/SC22/WG21/docs/lwg-active.html#530) +// proposes this as the method. According to Matt Austern, this should +// already work on all current implementations. +inline char* string_as_array(string* str) { + // DO NOT USE const_cast<char*>(str->data())! See the unittest for why. + return str->empty() ? NULL : &*str->begin(); +} + +// STLDeleteElements() deletes all the elements in an STL container and clears +// the container. This function is suitable for use with a vector, set, +// hash_set, or any other STL container which defines sensible begin(), end(), +// and clear() methods. +// +// If container is NULL, this function is a no-op. +// +// As an alternative to calling STLDeleteElements() directly, consider +// ElementDeleter (defined below), which ensures that your container's elements +// are deleted when the ElementDeleter goes out of scope. +template <class T> +void STLDeleteElements(T *container) { + if (!container) return; + STLDeleteContainerPointers(container->begin(), container->end()); + container->clear(); +} + +// Given an STL container consisting of (key, value) pairs, STLDeleteValues +// deletes all the "value" components and clears the container. Does nothing +// in the case it's given a NULL pointer. + +template <class T> +void STLDeleteValues(T *v) { + if (!v) return; + for (typename T::iterator i = v->begin(); i != v->end(); ++i) { + delete i->second; + } + v->clear(); +} + +} // namespace protobuf +} // namespace google + +#endif // GOOGLE_PROTOBUF_STUBS_STL_UTIL_H__ diff --git a/third_party/protobuf/src/google/protobuf/stubs/stringprintf.cc b/third_party/protobuf/src/google/protobuf/stubs/stringprintf.cc new file mode 100644 index 0000000..4a5b858 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/stringprintf.cc @@ -0,0 +1,175 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2012 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// from google3/base/stringprintf.cc + +#include <google/protobuf/stubs/stringprintf.h> + +#include <errno.h> +#include <stdarg.h> // For va_list and related operations +#include <stdio.h> // MSVC requires this for _vsnprintf +#include <vector> +#include <google/protobuf/stubs/common.h> +#include <google/protobuf/testing/googletest.h> + +namespace google { +namespace protobuf { + +#ifdef _MSC_VER +enum { IS_COMPILER_MSVC = 1 }; +#ifndef va_copy +// Define va_copy for MSVC. This is a hack, assuming va_list is simply a +// pointer into the stack and is safe to copy. +#define va_copy(dest, src) ((dest) = (src)) +#endif +#else +enum { IS_COMPILER_MSVC = 0 }; +#endif + +void StringAppendV(string* dst, const char* format, va_list ap) { + // First try with a small fixed size buffer + static const int kSpaceLength = 1024; + char space[kSpaceLength]; + + // It's possible for methods that use a va_list to invalidate + // the data in it upon use. The fix is to make a copy + // of the structure before using it and use that copy instead. + va_list backup_ap; + va_copy(backup_ap, ap); + int result = vsnprintf(space, kSpaceLength, format, backup_ap); + va_end(backup_ap); + + if (result < kSpaceLength) { + if (result >= 0) { + // Normal case -- everything fit. + dst->append(space, result); + return; + } + + if (IS_COMPILER_MSVC) { + // Error or MSVC running out of space. MSVC 8.0 and higher + // can be asked about space needed with the special idiom below: + va_copy(backup_ap, ap); + result = vsnprintf(NULL, 0, format, backup_ap); + va_end(backup_ap); + } + + if (result < 0) { + // Just an error. + return; + } + } + + // Increase the buffer size to the size requested by vsnprintf, + // plus one for the closing \0. + int length = result+1; + char* buf = new char[length]; + + // Restore the va_list before we use it again + va_copy(backup_ap, ap); + result = vsnprintf(buf, length, format, backup_ap); + va_end(backup_ap); + + if (result >= 0 && result < length) { + // It fit + dst->append(buf, result); + } + delete[] buf; +} + + +string StringPrintf(const char* format, ...) { + va_list ap; + va_start(ap, format); + string result; + StringAppendV(&result, format, ap); + va_end(ap); + return result; +} + +const string& SStringPrintf(string* dst, const char* format, ...) { + va_list ap; + va_start(ap, format); + dst->clear(); + StringAppendV(dst, format, ap); + va_end(ap); + return *dst; +} + +void StringAppendF(string* dst, const char* format, ...) { + va_list ap; + va_start(ap, format); + StringAppendV(dst, format, ap); + va_end(ap); +} + +// Max arguments supported by StringPrintVector +const int kStringPrintfVectorMaxArgs = 32; + +// An empty block of zero for filler arguments. This is const so that if +// printf tries to write to it (via %n) then the program gets a SIGSEGV +// and we can fix the problem or protect against an attack. +static const char string_printf_empty_block[256] = { '\0' }; + +string StringPrintfVector(const char* format, const vector<string>& v) { + GOOGLE_CHECK_LE(v.size(), kStringPrintfVectorMaxArgs) + << "StringPrintfVector currently only supports up to " + << kStringPrintfVectorMaxArgs << " arguments. " + << "Feel free to add support for more if you need it."; + + // Add filler arguments so that bogus format+args have a harder time + // crashing the program, corrupting the program (%n), + // or displaying random chunks of memory to users. + + const char* cstr[kStringPrintfVectorMaxArgs]; + for (int i = 0; i < v.size(); ++i) { + cstr[i] = v[i].c_str(); + } + for (int i = v.size(); i < GOOGLE_ARRAYSIZE(cstr); ++i) { + cstr[i] = &string_printf_empty_block[0]; + } + + // I do not know any way to pass kStringPrintfVectorMaxArgs arguments, + // or any way to build a va_list by hand, or any API for printf + // that accepts an array of arguments. The best I can do is stick + // this COMPILE_ASSERT right next to the actual statement. + + GOOGLE_COMPILE_ASSERT(kStringPrintfVectorMaxArgs == 32, arg_count_mismatch); + return StringPrintf(format, + cstr[0], cstr[1], cstr[2], cstr[3], cstr[4], + cstr[5], cstr[6], cstr[7], cstr[8], cstr[9], + cstr[10], cstr[11], cstr[12], cstr[13], cstr[14], + cstr[15], cstr[16], cstr[17], cstr[18], cstr[19], + cstr[20], cstr[21], cstr[22], cstr[23], cstr[24], + cstr[25], cstr[26], cstr[27], cstr[28], cstr[29], + cstr[30], cstr[31]); +} +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/stubs/stringprintf.h b/third_party/protobuf/src/google/protobuf/stubs/stringprintf.h new file mode 100644 index 0000000..573f30f --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/stringprintf.h @@ -0,0 +1,76 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2012 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// from google3/base/stringprintf.h +// +// Printf variants that place their output in a C++ string. +// +// Usage: +// string result = StringPrintf("%d %s\n", 10, "hello"); +// SStringPrintf(&result, "%d %s\n", 10, "hello"); +// StringAppendF(&result, "%d %s\n", 20, "there"); + +#ifndef GOOGLE_PROTOBUF_STUBS_STRINGPRINTF_H +#define GOOGLE_PROTOBUF_STUBS_STRINGPRINTF_H + +#include <stdarg.h> +#include <string> +#include <vector> + +#include <google/protobuf/stubs/common.h> + +namespace google { +namespace protobuf { + +// Return a C++ string +extern string StringPrintf(const char* format, ...); + +// Store result into a supplied string and return it +extern const string& SStringPrintf(string* dst, const char* format, ...); + +// Append result to a supplied string +extern void StringAppendF(string* dst, const char* format, ...); + +// Lower-level routine that takes a va_list and appends to a specified +// string. All other routines are just convenience wrappers around it. +extern void StringAppendV(string* dst, const char* format, va_list ap); + +// The max arguments supported by StringPrintfVector +extern const int kStringPrintfVectorMaxArgs; + +// You can use this version when all your arguments are strings, but +// you don't know how many arguments you'll have at compile time. +// StringPrintfVector will LOG(FATAL) if v.size() > kStringPrintfVectorMaxArgs +extern string StringPrintfVector(const char* format, const vector<string>& v); + +} // namespace protobuf +} // namespace google + +#endif // GOOGLE_PROTOBUF_STUBS_STRINGPRINTF_H diff --git a/third_party/protobuf/src/google/protobuf/stubs/stringprintf_unittest.cc b/third_party/protobuf/src/google/protobuf/stubs/stringprintf_unittest.cc new file mode 100644 index 0000000..a78a202 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/stringprintf_unittest.cc @@ -0,0 +1,152 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2012 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// from google3/base/stringprintf_unittest.cc + +#include <google/protobuf/stubs/stringprintf.h> + +#include <cerrno> +#include <string> + +#include <google/protobuf/testing/googletest.h> +#include <gtest/gtest.h> + +namespace google { +namespace protobuf { +namespace { + +TEST(StringPrintfTest, Empty) { +#if 0 + // gcc 2.95.3, gcc 4.1.0, and gcc 4.2.2 all warn about this: + // warning: zero-length printf format string. + // so we do not allow them in google3. + EXPECT_EQ("", StringPrintf("")); +#endif + EXPECT_EQ("", StringPrintf("%s", string().c_str())); + EXPECT_EQ("", StringPrintf("%s", "")); +} + +TEST(StringPrintfTest, Misc) { +// MSVC does not support $ format specifier. +#if !defined(_MSC_VER) + EXPECT_EQ("123hello w", StringPrintf("%3$d%2$s %1$c", 'w', "hello", 123)); +#endif // !_MSC_VER +} + +TEST(StringAppendFTest, Empty) { + string value("Hello"); + const char* empty = ""; + StringAppendF(&value, "%s", empty); + EXPECT_EQ("Hello", value); +} + +TEST(StringAppendFTest, EmptyString) { + string value("Hello"); + StringAppendF(&value, "%s", ""); + EXPECT_EQ("Hello", value); +} + +TEST(StringAppendFTest, String) { + string value("Hello"); + StringAppendF(&value, " %s", "World"); + EXPECT_EQ("Hello World", value); +} + +TEST(StringAppendFTest, Int) { + string value("Hello"); + StringAppendF(&value, " %d", 123); + EXPECT_EQ("Hello 123", value); +} + +TEST(StringPrintfTest, Multibyte) { + // If we are in multibyte mode and feed invalid multibyte sequence, + // StringPrintf should return an empty string instead of running + // out of memory while trying to determine destination buffer size. + // see b/4194543. + + char* old_locale = setlocale(LC_CTYPE, NULL); + // Push locale with multibyte mode + setlocale(LC_CTYPE, "en_US.utf8"); + + const char kInvalidCodePoint[] = "\375\067s"; + string value = StringPrintf("%.*s", 3, kInvalidCodePoint); + + // In some versions of glibc (e.g. eglibc-2.11.1, aka GRTEv2), snprintf + // returns error given an invalid codepoint. Other versions + // (e.g. eglibc-2.15, aka pre-GRTEv3) emit the codepoint verbatim. + // We test that the output is one of the above. + EXPECT_TRUE(value.empty() || value == kInvalidCodePoint); + + // Repeat with longer string, to make sure that the dynamically + // allocated path in StringAppendV is handled correctly. + int n = 2048; + char* buf = new char[n+1]; + memset(buf, ' ', n-3); + memcpy(buf + n - 3, kInvalidCodePoint, 4); + value = StringPrintf("%.*s", n, buf); + // See GRTEv2 vs. GRTEv3 comment above. + EXPECT_TRUE(value.empty() || value == buf); + delete[] buf; + + setlocale(LC_CTYPE, old_locale); +} + +TEST(StringPrintfTest, NoMultibyte) { + // No multibyte handling, but the string contains funny chars. + char* old_locale = setlocale(LC_CTYPE, NULL); + setlocale(LC_CTYPE, "POSIX"); + string value = StringPrintf("%.*s", 3, "\375\067s"); + setlocale(LC_CTYPE, old_locale); + EXPECT_EQ("\375\067s", value); +} + +TEST(StringPrintfTest, DontOverwriteErrno) { + // Check that errno isn't overwritten unless we're printing + // something significantly larger than what people are normally + // printing in their badly written PLOG() statements. + errno = ECHILD; + string value = StringPrintf("Hello, %s!", "World"); + EXPECT_EQ(ECHILD, errno); +} + +TEST(StringPrintfTest, LargeBuf) { + // Check that the large buffer is handled correctly. + int n = 2048; + char* buf = new char[n+1]; + memset(buf, ' ', n); + buf[n] = 0; + string value = StringPrintf("%s", buf); + EXPECT_EQ(buf, value); + delete[] buf; +} + +} // anonymous namespace +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/stubs/strutil.cc b/third_party/protobuf/src/google/protobuf/stubs/strutil.cc index ee07ce7..917b3e9 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/strutil.cc +++ b/third_party/protobuf/src/google/protobuf/stubs/strutil.cc @@ -189,6 +189,44 @@ void SplitStringUsing(const string& full, SplitStringToIteratorUsing(full, delim, it); } +// Split a string using a character delimiter. Append the components +// to 'result'. If there are consecutive delimiters, this function +// will return corresponding empty strings. The string is split into +// at most the specified number of pieces greedily. This means that the +// last piece may possibly be split further. To split into as many pieces +// as possible, specify 0 as the number of pieces. +// +// If "full" is the empty string, yields an empty string as the only value. +// +// If "pieces" is negative for some reason, it returns the whole string +// ---------------------------------------------------------------------- +template <typename StringType, typename ITR> +static inline +void SplitStringToIteratorAllowEmpty(const StringType& full, + const char* delim, + int pieces, + ITR& result) { + string::size_type begin_index, end_index; + begin_index = 0; + + for (int i = 0; (i < pieces-1) || (pieces == 0); i++) { + end_index = full.find_first_of(delim, begin_index); + if (end_index == string::npos) { + *result++ = full.substr(begin_index); + return; + } + *result++ = full.substr(begin_index, (end_index - begin_index)); + begin_index = end_index + 1; + } + *result++ = full.substr(begin_index); +} + +void SplitStringAllowEmpty(const string& full, const char* delim, + vector<string>* result) { + back_insert_iterator<vector<string> > it(*result); + SplitStringToIteratorAllowEmpty(full, delim, 0, it); +} + // ---------------------------------------------------------------------- // JoinStrings() // This merges a vector of string components with delim inserted diff --git a/third_party/protobuf/src/google/protobuf/stubs/strutil.h b/third_party/protobuf/src/google/protobuf/stubs/strutil.h index 4a79c22..a401c63 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/strutil.h +++ b/third_party/protobuf/src/google/protobuf/stubs/strutil.h @@ -168,6 +168,18 @@ LIBPROTOBUF_EXPORT string StringReplace(const string& s, const string& oldsub, LIBPROTOBUF_EXPORT void SplitStringUsing(const string& full, const char* delim, vector<string>* res); +// Split a string using one or more byte delimiters, presented +// as a nul-terminated c string. Append the components to 'result'. +// If there are consecutive delimiters, this function will return +// corresponding empty strings. If you want to drop the empty +// strings, try SplitStringUsing(). +// +// If "full" is the empty string, yields an empty string as the only value. +// ---------------------------------------------------------------------- +LIBPROTOBUF_EXPORT void SplitStringAllowEmpty(const string& full, + const char* delim, + vector<string>* result); + // ---------------------------------------------------------------------- // JoinStrings() // These methods concatenate a vector of strings into a C++ string, using @@ -453,5 +465,3 @@ LIBPROTOBUF_EXPORT double NoLocaleStrtod(const char* text, char** endptr); } // namespace google #endif // GOOGLE_PROTOBUF_STUBS_STRUTIL_H__ - - diff --git a/third_party/protobuf/src/google/protobuf/stubs/substitute.cc b/third_party/protobuf/src/google/protobuf/stubs/substitute.cc index b542aaa..259245b 100644 --- a/third_party/protobuf/src/google/protobuf/stubs/substitute.cc +++ b/third_party/protobuf/src/google/protobuf/stubs/substitute.cc @@ -32,7 +32,7 @@ #include <google/protobuf/stubs/substitute.h> #include <google/protobuf/stubs/strutil.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { diff --git a/third_party/protobuf/src/google/protobuf/stubs/template_util.h b/third_party/protobuf/src/google/protobuf/stubs/template_util.h new file mode 100644 index 0000000..4f30ffa --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/template_util.h @@ -0,0 +1,138 @@ +// Copyright 2005 Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// ---- +// Author: lar@google.com (Laramie Leavitt) +// +// Template metaprogramming utility functions. +// +// This code is compiled directly on many platforms, including client +// platforms like Windows, Mac, and embedded systems. Before making +// any changes here, make sure that you're not breaking any platforms. +// +// +// The names choosen here reflect those used in tr1 and the boost::mpl +// library, there are similar operations used in the Loki library as +// well. I prefer the boost names for 2 reasons: +// 1. I think that portions of the Boost libraries are more likely to +// be included in the c++ standard. +// 2. It is not impossible that some of the boost libraries will be +// included in our own build in the future. +// Both of these outcomes means that we may be able to directly replace +// some of these with boost equivalents. +// +#ifndef GOOGLE_PROTOBUF_TEMPLATE_UTIL_H_ +#define GOOGLE_PROTOBUF_TEMPLATE_UTIL_H_ + +namespace google { +namespace protobuf { +namespace internal { + +// Types small_ and big_ are guaranteed such that sizeof(small_) < +// sizeof(big_) +typedef char small_; + +struct big_ { + char dummy[2]; +}; + +// Identity metafunction. +template <class T> +struct identity_ { + typedef T type; +}; + +// integral_constant, defined in tr1, is a wrapper for an integer +// value. We don't really need this generality; we could get away +// with hardcoding the integer type to bool. We use the fully +// general integer_constant for compatibility with tr1. + +template<class T, T v> +struct integral_constant { + static const T value = v; + typedef T value_type; + typedef integral_constant<T, v> type; +}; + +template <class T, T v> const T integral_constant<T, v>::value; + + +// Abbreviations: true_type and false_type are structs that represent boolean +// true and false values. Also define the boost::mpl versions of those names, +// true_ and false_. +typedef integral_constant<bool, true> true_type; +typedef integral_constant<bool, false> false_type; +typedef true_type true_; +typedef false_type false_; + +// if_ is a templatized conditional statement. +// if_<cond, A, B> is a compile time evaluation of cond. +// if_<>::type contains A if cond is true, B otherwise. +template<bool cond, typename A, typename B> +struct if_{ + typedef A type; +}; + +template<typename A, typename B> +struct if_<false, A, B> { + typedef B type; +}; + + +// type_equals_ is a template type comparator, similar to Loki IsSameType. +// type_equals_<A, B>::value is true iff "A" is the same type as "B". +// +// New code should prefer base::is_same, defined in base/type_traits.h. +// It is functionally identical, but is_same is the standard spelling. +template<typename A, typename B> +struct type_equals_ : public false_ { +}; + +template<typename A> +struct type_equals_<A, A> : public true_ { +}; + +// and_ is a template && operator. +// and_<A, B>::value evaluates "A::value && B::value". +template<typename A, typename B> +struct and_ : public integral_constant<bool, (A::value && B::value)> { +}; + +// or_ is a template || operator. +// or_<A, B>::value evaluates "A::value || B::value". +template<typename A, typename B> +struct or_ : public integral_constant<bool, (A::value || B::value)> { +}; + + +} // namespace internal +} // namespace protobuf +} // namespace google + +#endif // GOOGLE_PROTOBUF_TEMPLATE_UTIL_H_ diff --git a/third_party/protobuf/src/google/protobuf/stubs/template_util_unittest.cc b/third_party/protobuf/src/google/protobuf/stubs/template_util_unittest.cc new file mode 100644 index 0000000..b1745e2 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/template_util_unittest.cc @@ -0,0 +1,130 @@ +// Copyright 2005 Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// ---- +// Author: lar@google.com (Laramie Leavitt) +// +// These tests are really compile time tests. +// If you try to step through this in a debugger +// you will not see any evaluations, merely that +// value is assigned true or false sequentially. + +#include <google/protobuf/stubs/template_util.h> + +#include <google/protobuf/testing/googletest.h> +#include <gtest/gtest.h> + +namespace GOOGLE_NAMESPACE = google::protobuf::internal; + +namespace google { +namespace protobuf { +namespace internal { +namespace { + +TEST(TemplateUtilTest, TestSize) { + EXPECT_GT(sizeof(GOOGLE_NAMESPACE::big_), sizeof(GOOGLE_NAMESPACE::small_)); +} + +TEST(TemplateUtilTest, TestIntegralConstants) { + // test the built-in types. + EXPECT_TRUE(true_type::value); + EXPECT_FALSE(false_type::value); + + typedef integral_constant<int, 1> one_type; + EXPECT_EQ(1, one_type::value); +} + +TEST(TemplateUtilTest, TestTemplateIf) { + typedef if_<true, true_type, false_type>::type if_true; + EXPECT_TRUE(if_true::value); + + typedef if_<false, true_type, false_type>::type if_false; + EXPECT_FALSE(if_false::value); +} + +TEST(TemplateUtilTest, TestTemplateTypeEquals) { + // Check that the TemplateTypeEquals works correctly. + bool value = false; + + // Test the same type is true. + value = type_equals_<int, int>::value; + EXPECT_TRUE(value); + + // Test different types are false. + value = type_equals_<float, int>::value; + EXPECT_FALSE(value); + + // Test type aliasing. + typedef const int foo; + value = type_equals_<const foo, const int>::value; + EXPECT_TRUE(value); +} + +TEST(TemplateUtilTest, TestTemplateAndOr) { + // Check that the TemplateTypeEquals works correctly. + bool value = false; + + // Yes && Yes == true. + value = and_<true_, true_>::value; + EXPECT_TRUE(value); + // Yes && No == false. + value = and_<true_, false_>::value; + EXPECT_FALSE(value); + // No && Yes == false. + value = and_<false_, true_>::value; + EXPECT_FALSE(value); + // No && No == false. + value = and_<false_, false_>::value; + EXPECT_FALSE(value); + + // Yes || Yes == true. + value = or_<true_, true_>::value; + EXPECT_TRUE(value); + // Yes || No == true. + value = or_<true_, false_>::value; + EXPECT_TRUE(value); + // No || Yes == true. + value = or_<false_, true_>::value; + EXPECT_TRUE(value); + // No || No == false. + value = or_<false_, false_>::value; + EXPECT_FALSE(value); +} + +TEST(TemplateUtilTest, TestIdentity) { + EXPECT_TRUE( + (type_equals_<GOOGLE_NAMESPACE::identity_<int>::type, int>::value)); + EXPECT_TRUE( + (type_equals_<GOOGLE_NAMESPACE::identity_<void>::type, void>::value)); +} + +} // anonymous namespace +} // namespace internal +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/stubs/type_traits.h b/third_party/protobuf/src/google/protobuf/stubs/type_traits.h new file mode 100644 index 0000000..997f35d --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/type_traits.h @@ -0,0 +1,346 @@ +// Copyright (c) 2006, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// ---- +// Author: Matt Austern +// +// This code is compiled directly on many platforms, including client +// platforms like Windows, Mac, and embedded systems. Before making +// any changes here, make sure that you're not breaking any platforms. +// +// Define a small subset of tr1 type traits. The traits we define are: +// is_integral +// is_floating_point +// is_pointer +// is_enum +// is_reference +// is_pod +// has_trivial_constructor +// has_trivial_copy +// has_trivial_assign +// has_trivial_destructor +// remove_const +// remove_volatile +// remove_cv +// remove_reference +// add_reference +// remove_pointer +// is_same +// is_convertible +// We can add more type traits as required. + +#ifndef GOOGLE_PROTOBUF_TYPE_TRAITS_H_ +#define GOOGLE_PROTOBUF_TYPE_TRAITS_H_ + +#include <utility> // For pair + +#include <google/protobuf/stubs/template_util.h> // For true_type and false_type + +namespace google { +namespace protobuf { +namespace internal { + +template <class T> struct is_integral; +template <class T> struct is_floating_point; +template <class T> struct is_pointer; +// MSVC can't compile this correctly, and neither can gcc 3.3.5 (at least) +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) +// is_enum uses is_convertible, which is not available on MSVC. +template <class T> struct is_enum; +#endif +template <class T> struct is_reference; +template <class T> struct is_pod; +template <class T> struct has_trivial_constructor; +template <class T> struct has_trivial_copy; +template <class T> struct has_trivial_assign; +template <class T> struct has_trivial_destructor; +template <class T> struct remove_const; +template <class T> struct remove_volatile; +template <class T> struct remove_cv; +template <class T> struct remove_reference; +template <class T> struct add_reference; +template <class T> struct remove_pointer; +template <class T, class U> struct is_same; +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) +template <class From, class To> struct is_convertible; +#endif + +// is_integral is false except for the built-in integer types. A +// cv-qualified type is integral if and only if the underlying type is. +template <class T> struct is_integral : false_type { }; +template<> struct is_integral<bool> : true_type { }; +template<> struct is_integral<char> : true_type { }; +template<> struct is_integral<unsigned char> : true_type { }; +template<> struct is_integral<signed char> : true_type { }; +#if defined(_MSC_VER) +// wchar_t is not by default a distinct type from unsigned short in +// Microsoft C. +// See http://msdn2.microsoft.com/en-us/library/dh8che7s(VS.80).aspx +template<> struct is_integral<__wchar_t> : true_type { }; +#else +template<> struct is_integral<wchar_t> : true_type { }; +#endif +template<> struct is_integral<short> : true_type { }; +template<> struct is_integral<unsigned short> : true_type { }; +template<> struct is_integral<int> : true_type { }; +template<> struct is_integral<unsigned int> : true_type { }; +template<> struct is_integral<long> : true_type { }; +template<> struct is_integral<unsigned long> : true_type { }; +#ifdef HAVE_LONG_LONG +template<> struct is_integral<long long> : true_type { }; +template<> struct is_integral<unsigned long long> : true_type { }; +#endif +template <class T> struct is_integral<const T> : is_integral<T> { }; +template <class T> struct is_integral<volatile T> : is_integral<T> { }; +template <class T> struct is_integral<const volatile T> : is_integral<T> { }; + +// is_floating_point is false except for the built-in floating-point types. +// A cv-qualified type is integral if and only if the underlying type is. +template <class T> struct is_floating_point : false_type { }; +template<> struct is_floating_point<float> : true_type { }; +template<> struct is_floating_point<double> : true_type { }; +template<> struct is_floating_point<long double> : true_type { }; +template <class T> struct is_floating_point<const T> + : is_floating_point<T> { }; +template <class T> struct is_floating_point<volatile T> + : is_floating_point<T> { }; +template <class T> struct is_floating_point<const volatile T> + : is_floating_point<T> { }; + +// is_pointer is false except for pointer types. A cv-qualified type (e.g. +// "int* const", as opposed to "int const*") is cv-qualified if and only if +// the underlying type is. +template <class T> struct is_pointer : false_type { }; +template <class T> struct is_pointer<T*> : true_type { }; +template <class T> struct is_pointer<const T> : is_pointer<T> { }; +template <class T> struct is_pointer<volatile T> : is_pointer<T> { }; +template <class T> struct is_pointer<const volatile T> : is_pointer<T> { }; + +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) + +namespace internal { + +template <class T> struct is_class_or_union { + template <class U> static small_ tester(void (U::*)()); + template <class U> static big_ tester(...); + static const bool value = sizeof(tester<T>(0)) == sizeof(small_); +}; + +// is_convertible chokes if the first argument is an array. That's why +// we use add_reference here. +template <bool NotUnum, class T> struct is_enum_impl + : is_convertible<typename add_reference<T>::type, int> { }; + +template <class T> struct is_enum_impl<true, T> : false_type { }; + +} // namespace internal + +// Specified by TR1 [4.5.1] primary type categories. + +// Implementation note: +// +// Each type is either void, integral, floating point, array, pointer, +// reference, member object pointer, member function pointer, enum, +// union or class. Out of these, only integral, floating point, reference, +// class and enum types are potentially convertible to int. Therefore, +// if a type is not a reference, integral, floating point or class and +// is convertible to int, it's a enum. Adding cv-qualification to a type +// does not change whether it's an enum. +// +// Is-convertible-to-int check is done only if all other checks pass, +// because it can't be used with some types (e.g. void or classes with +// inaccessible conversion operators). +template <class T> struct is_enum + : internal::is_enum_impl< + is_same<T, void>::value || + is_integral<T>::value || + is_floating_point<T>::value || + is_reference<T>::value || + internal::is_class_or_union<T>::value, + T> { }; + +template <class T> struct is_enum<const T> : is_enum<T> { }; +template <class T> struct is_enum<volatile T> : is_enum<T> { }; +template <class T> struct is_enum<const volatile T> : is_enum<T> { }; + +#endif + +// is_reference is false except for reference types. +template<typename T> struct is_reference : false_type {}; +template<typename T> struct is_reference<T&> : true_type {}; + + +// We can't get is_pod right without compiler help, so fail conservatively. +// We will assume it's false except for arithmetic types, enumerations, +// pointers and cv-qualified versions thereof. Note that std::pair<T,U> +// is not a POD even if T and U are PODs. +template <class T> struct is_pod + : integral_constant<bool, (is_integral<T>::value || + is_floating_point<T>::value || +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) + // is_enum is not available on MSVC. + is_enum<T>::value || +#endif + is_pointer<T>::value)> { }; +template <class T> struct is_pod<const T> : is_pod<T> { }; +template <class T> struct is_pod<volatile T> : is_pod<T> { }; +template <class T> struct is_pod<const volatile T> : is_pod<T> { }; + + +// We can't get has_trivial_constructor right without compiler help, so +// fail conservatively. We will assume it's false except for: (1) types +// for which is_pod is true. (2) std::pair of types with trivial +// constructors. (3) array of a type with a trivial constructor. +// (4) const versions thereof. +template <class T> struct has_trivial_constructor : is_pod<T> { }; +template <class T, class U> struct has_trivial_constructor<std::pair<T, U> > + : integral_constant<bool, + (has_trivial_constructor<T>::value && + has_trivial_constructor<U>::value)> { }; +template <class A, int N> struct has_trivial_constructor<A[N]> + : has_trivial_constructor<A> { }; +template <class T> struct has_trivial_constructor<const T> + : has_trivial_constructor<T> { }; + +// We can't get has_trivial_copy right without compiler help, so fail +// conservatively. We will assume it's false except for: (1) types +// for which is_pod is true. (2) std::pair of types with trivial copy +// constructors. (3) array of a type with a trivial copy constructor. +// (4) const versions thereof. +template <class T> struct has_trivial_copy : is_pod<T> { }; +template <class T, class U> struct has_trivial_copy<std::pair<T, U> > + : integral_constant<bool, + (has_trivial_copy<T>::value && + has_trivial_copy<U>::value)> { }; +template <class A, int N> struct has_trivial_copy<A[N]> + : has_trivial_copy<A> { }; +template <class T> struct has_trivial_copy<const T> : has_trivial_copy<T> { }; + +// We can't get has_trivial_assign right without compiler help, so fail +// conservatively. We will assume it's false except for: (1) types +// for which is_pod is true. (2) std::pair of types with trivial copy +// constructors. (3) array of a type with a trivial assign constructor. +template <class T> struct has_trivial_assign : is_pod<T> { }; +template <class T, class U> struct has_trivial_assign<std::pair<T, U> > + : integral_constant<bool, + (has_trivial_assign<T>::value && + has_trivial_assign<U>::value)> { }; +template <class A, int N> struct has_trivial_assign<A[N]> + : has_trivial_assign<A> { }; + +// We can't get has_trivial_destructor right without compiler help, so +// fail conservatively. We will assume it's false except for: (1) types +// for which is_pod is true. (2) std::pair of types with trivial +// destructors. (3) array of a type with a trivial destructor. +// (4) const versions thereof. +template <class T> struct has_trivial_destructor : is_pod<T> { }; +template <class T, class U> struct has_trivial_destructor<std::pair<T, U> > + : integral_constant<bool, + (has_trivial_destructor<T>::value && + has_trivial_destructor<U>::value)> { }; +template <class A, int N> struct has_trivial_destructor<A[N]> + : has_trivial_destructor<A> { }; +template <class T> struct has_trivial_destructor<const T> + : has_trivial_destructor<T> { }; + +// Specified by TR1 [4.7.1] +template<typename T> struct remove_const { typedef T type; }; +template<typename T> struct remove_const<T const> { typedef T type; }; +template<typename T> struct remove_volatile { typedef T type; }; +template<typename T> struct remove_volatile<T volatile> { typedef T type; }; +template<typename T> struct remove_cv { + typedef typename remove_const<typename remove_volatile<T>::type>::type type; +}; + + +// Specified by TR1 [4.7.2] Reference modifications. +template<typename T> struct remove_reference { typedef T type; }; +template<typename T> struct remove_reference<T&> { typedef T type; }; + +template <typename T> struct add_reference { typedef T& type; }; +template <typename T> struct add_reference<T&> { typedef T& type; }; + +// Specified by TR1 [4.7.4] Pointer modifications. +template<typename T> struct remove_pointer { typedef T type; }; +template<typename T> struct remove_pointer<T*> { typedef T type; }; +template<typename T> struct remove_pointer<T* const> { typedef T type; }; +template<typename T> struct remove_pointer<T* volatile> { typedef T type; }; +template<typename T> struct remove_pointer<T* const volatile> { + typedef T type; }; + +// Specified by TR1 [4.6] Relationships between types +template<typename T, typename U> struct is_same : public false_type { }; +template<typename T> struct is_same<T, T> : public true_type { }; + +// Specified by TR1 [4.6] Relationships between types +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) +namespace internal { + +// This class is an implementation detail for is_convertible, and you +// don't need to know how it works to use is_convertible. For those +// who care: we declare two different functions, one whose argument is +// of type To and one with a variadic argument list. We give them +// return types of different size, so we can use sizeof to trick the +// compiler into telling us which function it would have chosen if we +// had called it with an argument of type From. See Alexandrescu's +// _Modern C++ Design_ for more details on this sort of trick. + +template <typename From, typename To> +struct ConvertHelper { + static small_ Test(To); + static big_ Test(...); + static From Create(); +}; +} // namespace internal + +// Inherits from true_type if From is convertible to To, false_type otherwise. +template <typename From, typename To> +struct is_convertible + : integral_constant<bool, + sizeof(internal::ConvertHelper<From, To>::Test( + internal::ConvertHelper<From, To>::Create())) + == sizeof(small_)> { +}; +#endif + +} // namespace internal +} // namespace protobuf +} // namespace google + +// Right now these macros are no-ops, and mostly just document the fact +// these types are PODs, for human use. They may be made more contentful +// later. The typedef is just to make it legal to put a semicolon after +// these macros. +#define DECLARE_POD(TypeName) typedef int Dummy_Type_For_DECLARE_POD +#define DECLARE_NESTED_POD(TypeName) DECLARE_POD(TypeName) +#define PROPAGATE_POD_FROM_TEMPLATE_ARGUMENT(TemplateName) \ + typedef int Dummy_Type_For_PROPAGATE_POD_FROM_TEMPLATE_ARGUMENT +#define ENFORCE_POD(TypeName) typedef int Dummy_Type_For_ENFORCE_POD + +#endif // GOOGLE_PROTOBUF_TYPE_TRAITS_H_ diff --git a/third_party/protobuf/src/google/protobuf/stubs/type_traits_unittest.cc b/third_party/protobuf/src/google/protobuf/stubs/type_traits_unittest.cc new file mode 100644 index 0000000..00e5a47 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/stubs/type_traits_unittest.cc @@ -0,0 +1,647 @@ +// Copyright (c) 2006, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// ---- +// Author: Matt Austern + +#include <google/protobuf/stubs/type_traits.h> + +#include <stdlib.h> // for exit() +#include <stdio.h> +#include <string> +#include <vector> + +#include <google/protobuf/testing/googletest.h> +#include <gtest/gtest.h> + +typedef int int32; +typedef long int64; + +using std::string; +using std::vector; +using std::pair; + +using google::protobuf::internal::add_reference; +using google::protobuf::internal::has_trivial_assign; +using google::protobuf::internal::has_trivial_constructor; +using google::protobuf::internal::has_trivial_copy; +using google::protobuf::internal::has_trivial_destructor; +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) +using google::protobuf::internal::is_convertible; +using google::protobuf::internal::is_enum; +#endif +using google::protobuf::internal::is_floating_point; +using google::protobuf::internal::is_integral; +using google::protobuf::internal::is_pointer; +using google::protobuf::internal::is_pod; +using google::protobuf::internal::is_reference; +using google::protobuf::internal::is_same; +using google::protobuf::internal::remove_const; +using google::protobuf::internal::remove_cv; +using google::protobuf::internal::remove_pointer; +using google::protobuf::internal::remove_reference; +using google::protobuf::internal::remove_volatile; + + +// This assertion produces errors like "error: invalid use of +// incomplete type 'struct <unnamed>::AssertTypesEq<const int, int>'" +// when it fails. +template<typename T, typename U> struct AssertTypesEq; +template<typename T> struct AssertTypesEq<T, T> {}; +#define COMPILE_ASSERT_TYPES_EQ(T, U) static_cast<void>(AssertTypesEq<T, U>()) + +// A user-defined POD type. +struct A { + int n_; +}; + +// A user-defined non-POD type with a trivial copy constructor. +class B { + public: + explicit B(int n) : n_(n) { } + private: + int n_; +}; + +// Another user-defined non-POD type with a trivial copy constructor. +// We will explicitly declare C to have a trivial copy constructor +// by specializing has_trivial_copy. +class C { + public: + explicit C(int n) : n_(n) { } + private: + int n_; +}; + +namespace google { +namespace protobuf { +namespace internal { +template<> struct has_trivial_copy<C> : true_type { }; +} // namespace internal +} // namespace protobuf +} // namespace google + +// Another user-defined non-POD type with a trivial assignment operator. +// We will explicitly declare C to have a trivial assignment operator +// by specializing has_trivial_assign. +class D { + public: + explicit D(int n) : n_(n) { } + private: + int n_; +}; + +namespace google { +namespace protobuf { +namespace internal { +template<> struct has_trivial_assign<D> : true_type { }; +} // namespace internal +} // namespace protobuf +} // namespace google + +// Another user-defined non-POD type with a trivial constructor. +// We will explicitly declare E to have a trivial constructor +// by specializing has_trivial_constructor. +class E { + public: + int n_; +}; + +namespace google { +namespace protobuf { +namespace internal { +template<> struct has_trivial_constructor<E> : true_type { }; +} // namespace internal +} // namespace protobuf +} // namespace google + +// Another user-defined non-POD type with a trivial destructor. +// We will explicitly declare E to have a trivial destructor +// by specializing has_trivial_destructor. +class F { + public: + explicit F(int n) : n_(n) { } + private: + int n_; +}; + +namespace google { +namespace protobuf { +namespace internal { +template<> struct has_trivial_destructor<F> : true_type { }; +} // namespace internal +} // namespace protobuf +} // namespace google + +enum G {}; + +union H {}; + +class I { + public: + operator int() const; +}; + +class J { + private: + operator int() const; +}; + +namespace google { +namespace protobuf { +namespace { + +// A base class and a derived class that inherits from it, used for +// testing conversion type traits. +class Base { + public: + virtual ~Base() { } +}; + +class Derived : public Base { +}; + +TEST(TypeTraitsTest, TestIsInteger) { + // Verify that is_integral is true for all integer types. + EXPECT_TRUE(is_integral<bool>::value); + EXPECT_TRUE(is_integral<char>::value); + EXPECT_TRUE(is_integral<unsigned char>::value); + EXPECT_TRUE(is_integral<signed char>::value); + EXPECT_TRUE(is_integral<wchar_t>::value); + EXPECT_TRUE(is_integral<int>::value); + EXPECT_TRUE(is_integral<unsigned int>::value); + EXPECT_TRUE(is_integral<short>::value); + EXPECT_TRUE(is_integral<unsigned short>::value); + EXPECT_TRUE(is_integral<long>::value); + EXPECT_TRUE(is_integral<unsigned long>::value); + + // Verify that is_integral is false for a few non-integer types. + EXPECT_FALSE(is_integral<void>::value); + EXPECT_FALSE(is_integral<float>::value); + EXPECT_FALSE(is_integral<string>::value); + EXPECT_FALSE(is_integral<int*>::value); + EXPECT_FALSE(is_integral<A>::value); + EXPECT_FALSE((is_integral<pair<int, int> >::value)); + + // Verify that cv-qualified integral types are still integral, and + // cv-qualified non-integral types are still non-integral. + EXPECT_TRUE(is_integral<const char>::value); + EXPECT_TRUE(is_integral<volatile bool>::value); + EXPECT_TRUE(is_integral<const volatile unsigned int>::value); + EXPECT_FALSE(is_integral<const float>::value); + EXPECT_FALSE(is_integral<int* volatile>::value); + EXPECT_FALSE(is_integral<const volatile string>::value); +} + +TEST(TypeTraitsTest, TestIsFloating) { + // Verify that is_floating_point is true for all floating-point types. + EXPECT_TRUE(is_floating_point<float>::value); + EXPECT_TRUE(is_floating_point<double>::value); + EXPECT_TRUE(is_floating_point<long double>::value); + + // Verify that is_floating_point is false for a few non-float types. + EXPECT_FALSE(is_floating_point<void>::value); + EXPECT_FALSE(is_floating_point<long>::value); + EXPECT_FALSE(is_floating_point<string>::value); + EXPECT_FALSE(is_floating_point<float*>::value); + EXPECT_FALSE(is_floating_point<A>::value); + EXPECT_FALSE((is_floating_point<pair<int, int> >::value)); + + // Verify that cv-qualified floating point types are still floating, and + // cv-qualified non-floating types are still non-floating. + EXPECT_TRUE(is_floating_point<const float>::value); + EXPECT_TRUE(is_floating_point<volatile double>::value); + EXPECT_TRUE(is_floating_point<const volatile long double>::value); + EXPECT_FALSE(is_floating_point<const int>::value); + EXPECT_FALSE(is_floating_point<volatile string>::value); + EXPECT_FALSE(is_floating_point<const volatile char>::value); +} + +TEST(TypeTraitsTest, TestIsPointer) { + // Verify that is_pointer is true for some pointer types. + EXPECT_TRUE(is_pointer<int*>::value); + EXPECT_TRUE(is_pointer<void*>::value); + EXPECT_TRUE(is_pointer<string*>::value); + EXPECT_TRUE(is_pointer<const void*>::value); + EXPECT_TRUE(is_pointer<volatile float* const*>::value); + + // Verify that is_pointer is false for some non-pointer types. + EXPECT_FALSE(is_pointer<void>::value); + EXPECT_FALSE(is_pointer<float&>::value); + EXPECT_FALSE(is_pointer<long>::value); + EXPECT_FALSE(is_pointer<vector<int*> >::value); + EXPECT_FALSE(is_pointer<int[5]>::value); + + // A function pointer is a pointer, but a function type, or a function + // reference type, is not. + EXPECT_TRUE(is_pointer<int (*)(int x)>::value); + EXPECT_FALSE(is_pointer<void(char x)>::value); + EXPECT_FALSE(is_pointer<double (&)(string x)>::value); + + // Verify that is_pointer<T> is true for some cv-qualified pointer types, + // and false for some cv-qualified non-pointer types. + EXPECT_TRUE(is_pointer<int* const>::value); + EXPECT_TRUE(is_pointer<const void* volatile>::value); + EXPECT_TRUE(is_pointer<char** const volatile>::value); + EXPECT_FALSE(is_pointer<const int>::value); + EXPECT_FALSE(is_pointer<volatile vector<int*> >::value); + EXPECT_FALSE(is_pointer<const volatile double>::value); +} + +TEST(TypeTraitsTest, TestIsEnum) { +// is_enum isn't supported on MSVC or gcc 3.x +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) + // Verify that is_enum is true for enum types. + EXPECT_TRUE(is_enum<G>::value); + EXPECT_TRUE(is_enum<const G>::value); + EXPECT_TRUE(is_enum<volatile G>::value); + EXPECT_TRUE(is_enum<const volatile G>::value); + + // Verify that is_enum is false for a few non-enum types. + EXPECT_FALSE(is_enum<void>::value); + EXPECT_FALSE(is_enum<G&>::value); + EXPECT_FALSE(is_enum<G[1]>::value); + EXPECT_FALSE(is_enum<const G[1]>::value); + EXPECT_FALSE(is_enum<G[]>::value); + EXPECT_FALSE(is_enum<int>::value); + EXPECT_FALSE(is_enum<float>::value); + EXPECT_FALSE(is_enum<A>::value); + EXPECT_FALSE(is_enum<A*>::value); + EXPECT_FALSE(is_enum<const A>::value); + EXPECT_FALSE(is_enum<H>::value); + EXPECT_FALSE(is_enum<I>::value); + EXPECT_FALSE(is_enum<J>::value); + EXPECT_FALSE(is_enum<void()>::value); + EXPECT_FALSE(is_enum<void(*)()>::value); + EXPECT_FALSE(is_enum<int A::*>::value); + EXPECT_FALSE(is_enum<void (A::*)()>::value); +#endif +} + +TEST(TypeTraitsTest, TestIsReference) { + // Verifies that is_reference is true for all reference types. + typedef float& RefFloat; + EXPECT_TRUE(is_reference<float&>::value); + EXPECT_TRUE(is_reference<const int&>::value); + EXPECT_TRUE(is_reference<const int*&>::value); + EXPECT_TRUE(is_reference<int (&)(bool)>::value); + EXPECT_TRUE(is_reference<RefFloat>::value); + EXPECT_TRUE(is_reference<const RefFloat>::value); + EXPECT_TRUE(is_reference<volatile RefFloat>::value); + EXPECT_TRUE(is_reference<const volatile RefFloat>::value); + + + // Verifies that is_reference is false for all non-reference types. + EXPECT_FALSE(is_reference<float>::value); + EXPECT_FALSE(is_reference<const float>::value); + EXPECT_FALSE(is_reference<volatile float>::value); + EXPECT_FALSE(is_reference<const volatile float>::value); + EXPECT_FALSE(is_reference<const int*>::value); + EXPECT_FALSE(is_reference<int()>::value); + EXPECT_FALSE(is_reference<void(*)(const char&)>::value); +} + +TEST(TypeTraitsTest, TestAddReference) { + COMPILE_ASSERT_TYPES_EQ(int&, add_reference<int>::type); + COMPILE_ASSERT_TYPES_EQ(const int&, add_reference<const int>::type); + COMPILE_ASSERT_TYPES_EQ(volatile int&, + add_reference<volatile int>::type); + COMPILE_ASSERT_TYPES_EQ(const volatile int&, + add_reference<const volatile int>::type); + COMPILE_ASSERT_TYPES_EQ(int&, add_reference<int&>::type); + COMPILE_ASSERT_TYPES_EQ(const int&, add_reference<const int&>::type); + COMPILE_ASSERT_TYPES_EQ(volatile int&, + add_reference<volatile int&>::type); + COMPILE_ASSERT_TYPES_EQ(const volatile int&, + add_reference<const volatile int&>::type); +} + +TEST(TypeTraitsTest, TestIsPod) { + // Verify that arithmetic types and pointers are marked as PODs. + EXPECT_TRUE(is_pod<bool>::value); + EXPECT_TRUE(is_pod<char>::value); + EXPECT_TRUE(is_pod<unsigned char>::value); + EXPECT_TRUE(is_pod<signed char>::value); + EXPECT_TRUE(is_pod<wchar_t>::value); + EXPECT_TRUE(is_pod<int>::value); + EXPECT_TRUE(is_pod<unsigned int>::value); + EXPECT_TRUE(is_pod<short>::value); + EXPECT_TRUE(is_pod<unsigned short>::value); + EXPECT_TRUE(is_pod<long>::value); + EXPECT_TRUE(is_pod<unsigned long>::value); + EXPECT_TRUE(is_pod<float>::value); + EXPECT_TRUE(is_pod<double>::value); + EXPECT_TRUE(is_pod<long double>::value); + EXPECT_TRUE(is_pod<string*>::value); + EXPECT_TRUE(is_pod<A*>::value); + EXPECT_TRUE(is_pod<const B*>::value); + EXPECT_TRUE(is_pod<C**>::value); + EXPECT_TRUE(is_pod<const int>::value); + EXPECT_TRUE(is_pod<char* volatile>::value); + EXPECT_TRUE(is_pod<const volatile double>::value); +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) + EXPECT_TRUE(is_pod<G>::value); + EXPECT_TRUE(is_pod<const G>::value); + EXPECT_TRUE(is_pod<volatile G>::value); + EXPECT_TRUE(is_pod<const volatile G>::value); +#endif + + // Verify that some non-POD types are not marked as PODs. + EXPECT_FALSE(is_pod<void>::value); + EXPECT_FALSE(is_pod<string>::value); + EXPECT_FALSE((is_pod<pair<int, int> >::value)); + EXPECT_FALSE(is_pod<A>::value); + EXPECT_FALSE(is_pod<B>::value); + EXPECT_FALSE(is_pod<C>::value); + EXPECT_FALSE(is_pod<const string>::value); + EXPECT_FALSE(is_pod<volatile A>::value); + EXPECT_FALSE(is_pod<const volatile B>::value); +} + +TEST(TypeTraitsTest, TestHasTrivialConstructor) { + // Verify that arithmetic types and pointers have trivial constructors. + EXPECT_TRUE(has_trivial_constructor<bool>::value); + EXPECT_TRUE(has_trivial_constructor<char>::value); + EXPECT_TRUE(has_trivial_constructor<unsigned char>::value); + EXPECT_TRUE(has_trivial_constructor<signed char>::value); + EXPECT_TRUE(has_trivial_constructor<wchar_t>::value); + EXPECT_TRUE(has_trivial_constructor<int>::value); + EXPECT_TRUE(has_trivial_constructor<unsigned int>::value); + EXPECT_TRUE(has_trivial_constructor<short>::value); + EXPECT_TRUE(has_trivial_constructor<unsigned short>::value); + EXPECT_TRUE(has_trivial_constructor<long>::value); + EXPECT_TRUE(has_trivial_constructor<unsigned long>::value); + EXPECT_TRUE(has_trivial_constructor<float>::value); + EXPECT_TRUE(has_trivial_constructor<double>::value); + EXPECT_TRUE(has_trivial_constructor<long double>::value); + EXPECT_TRUE(has_trivial_constructor<string*>::value); + EXPECT_TRUE(has_trivial_constructor<A*>::value); + EXPECT_TRUE(has_trivial_constructor<const B*>::value); + EXPECT_TRUE(has_trivial_constructor<C**>::value); + + // Verify that pairs and arrays of such types have trivial + // constructors. + typedef int int10[10]; + EXPECT_TRUE((has_trivial_constructor<pair<int, char*> >::value)); + EXPECT_TRUE(has_trivial_constructor<int10>::value); + + // Verify that pairs of types without trivial constructors + // are not marked as trivial. + EXPECT_FALSE((has_trivial_constructor<pair<int, string> >::value)); + EXPECT_FALSE((has_trivial_constructor<pair<string, int> >::value)); + + // Verify that types without trivial constructors are + // correctly marked as such. + EXPECT_FALSE(has_trivial_constructor<string>::value); + EXPECT_FALSE(has_trivial_constructor<vector<int> >::value); + + // Verify that E, which we have declared to have a trivial + // constructor, is correctly marked as such. + EXPECT_TRUE(has_trivial_constructor<E>::value); +} + +TEST(TypeTraitsTest, TestHasTrivialCopy) { + // Verify that arithmetic types and pointers have trivial copy + // constructors. + EXPECT_TRUE(has_trivial_copy<bool>::value); + EXPECT_TRUE(has_trivial_copy<char>::value); + EXPECT_TRUE(has_trivial_copy<unsigned char>::value); + EXPECT_TRUE(has_trivial_copy<signed char>::value); + EXPECT_TRUE(has_trivial_copy<wchar_t>::value); + EXPECT_TRUE(has_trivial_copy<int>::value); + EXPECT_TRUE(has_trivial_copy<unsigned int>::value); + EXPECT_TRUE(has_trivial_copy<short>::value); + EXPECT_TRUE(has_trivial_copy<unsigned short>::value); + EXPECT_TRUE(has_trivial_copy<long>::value); + EXPECT_TRUE(has_trivial_copy<unsigned long>::value); + EXPECT_TRUE(has_trivial_copy<float>::value); + EXPECT_TRUE(has_trivial_copy<double>::value); + EXPECT_TRUE(has_trivial_copy<long double>::value); + EXPECT_TRUE(has_trivial_copy<string*>::value); + EXPECT_TRUE(has_trivial_copy<A*>::value); + EXPECT_TRUE(has_trivial_copy<const B*>::value); + EXPECT_TRUE(has_trivial_copy<C**>::value); + + // Verify that pairs and arrays of such types have trivial + // copy constructors. + typedef int int10[10]; + EXPECT_TRUE((has_trivial_copy<pair<int, char*> >::value)); + EXPECT_TRUE(has_trivial_copy<int10>::value); + + // Verify that pairs of types without trivial copy constructors + // are not marked as trivial. + EXPECT_FALSE((has_trivial_copy<pair<int, string> >::value)); + EXPECT_FALSE((has_trivial_copy<pair<string, int> >::value)); + + // Verify that types without trivial copy constructors are + // correctly marked as such. + EXPECT_FALSE(has_trivial_copy<string>::value); + EXPECT_FALSE(has_trivial_copy<vector<int> >::value); + + // Verify that C, which we have declared to have a trivial + // copy constructor, is correctly marked as such. + EXPECT_TRUE(has_trivial_copy<C>::value); +} + +TEST(TypeTraitsTest, TestHasTrivialAssign) { + // Verify that arithmetic types and pointers have trivial assignment + // operators. + EXPECT_TRUE(has_trivial_assign<bool>::value); + EXPECT_TRUE(has_trivial_assign<char>::value); + EXPECT_TRUE(has_trivial_assign<unsigned char>::value); + EXPECT_TRUE(has_trivial_assign<signed char>::value); + EXPECT_TRUE(has_trivial_assign<wchar_t>::value); + EXPECT_TRUE(has_trivial_assign<int>::value); + EXPECT_TRUE(has_trivial_assign<unsigned int>::value); + EXPECT_TRUE(has_trivial_assign<short>::value); + EXPECT_TRUE(has_trivial_assign<unsigned short>::value); + EXPECT_TRUE(has_trivial_assign<long>::value); + EXPECT_TRUE(has_trivial_assign<unsigned long>::value); + EXPECT_TRUE(has_trivial_assign<float>::value); + EXPECT_TRUE(has_trivial_assign<double>::value); + EXPECT_TRUE(has_trivial_assign<long double>::value); + EXPECT_TRUE(has_trivial_assign<string*>::value); + EXPECT_TRUE(has_trivial_assign<A*>::value); + EXPECT_TRUE(has_trivial_assign<const B*>::value); + EXPECT_TRUE(has_trivial_assign<C**>::value); + + // Verify that pairs and arrays of such types have trivial + // assignment operators. + typedef int int10[10]; + EXPECT_TRUE((has_trivial_assign<pair<int, char*> >::value)); + EXPECT_TRUE(has_trivial_assign<int10>::value); + + // Verify that pairs of types without trivial assignment operators + // are not marked as trivial. + EXPECT_FALSE((has_trivial_assign<pair<int, string> >::value)); + EXPECT_FALSE((has_trivial_assign<pair<string, int> >::value)); + + // Verify that types without trivial assignment operators are + // correctly marked as such. + EXPECT_FALSE(has_trivial_assign<string>::value); + EXPECT_FALSE(has_trivial_assign<vector<int> >::value); + + // Verify that D, which we have declared to have a trivial + // assignment operator, is correctly marked as such. + EXPECT_TRUE(has_trivial_assign<D>::value); +} + +TEST(TypeTraitsTest, TestHasTrivialDestructor) { + // Verify that arithmetic types and pointers have trivial destructors. + EXPECT_TRUE(has_trivial_destructor<bool>::value); + EXPECT_TRUE(has_trivial_destructor<char>::value); + EXPECT_TRUE(has_trivial_destructor<unsigned char>::value); + EXPECT_TRUE(has_trivial_destructor<signed char>::value); + EXPECT_TRUE(has_trivial_destructor<wchar_t>::value); + EXPECT_TRUE(has_trivial_destructor<int>::value); + EXPECT_TRUE(has_trivial_destructor<unsigned int>::value); + EXPECT_TRUE(has_trivial_destructor<short>::value); + EXPECT_TRUE(has_trivial_destructor<unsigned short>::value); + EXPECT_TRUE(has_trivial_destructor<long>::value); + EXPECT_TRUE(has_trivial_destructor<unsigned long>::value); + EXPECT_TRUE(has_trivial_destructor<float>::value); + EXPECT_TRUE(has_trivial_destructor<double>::value); + EXPECT_TRUE(has_trivial_destructor<long double>::value); + EXPECT_TRUE(has_trivial_destructor<string*>::value); + EXPECT_TRUE(has_trivial_destructor<A*>::value); + EXPECT_TRUE(has_trivial_destructor<const B*>::value); + EXPECT_TRUE(has_trivial_destructor<C**>::value); + + // Verify that pairs and arrays of such types have trivial + // destructors. + typedef int int10[10]; + EXPECT_TRUE((has_trivial_destructor<pair<int, char*> >::value)); + EXPECT_TRUE(has_trivial_destructor<int10>::value); + + // Verify that pairs of types without trivial destructors + // are not marked as trivial. + EXPECT_FALSE((has_trivial_destructor<pair<int, string> >::value)); + EXPECT_FALSE((has_trivial_destructor<pair<string, int> >::value)); + + // Verify that types without trivial destructors are + // correctly marked as such. + EXPECT_FALSE(has_trivial_destructor<string>::value); + EXPECT_FALSE(has_trivial_destructor<vector<int> >::value); + + // Verify that F, which we have declared to have a trivial + // destructor, is correctly marked as such. + EXPECT_TRUE(has_trivial_destructor<F>::value); +} + +// Tests remove_pointer. +TEST(TypeTraitsTest, TestRemovePointer) { + COMPILE_ASSERT_TYPES_EQ(int, remove_pointer<int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_pointer<int*>::type); + COMPILE_ASSERT_TYPES_EQ(const int, remove_pointer<const int*>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_pointer<int* const>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_pointer<int* volatile>::type); +} + +TEST(TypeTraitsTest, TestRemoveConst) { + COMPILE_ASSERT_TYPES_EQ(int, remove_const<int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_const<const int>::type); + COMPILE_ASSERT_TYPES_EQ(int *, remove_const<int * const>::type); + // TR1 examples. + COMPILE_ASSERT_TYPES_EQ(const int *, remove_const<const int *>::type); + COMPILE_ASSERT_TYPES_EQ(volatile int, + remove_const<const volatile int>::type); +} + +TEST(TypeTraitsTest, TestRemoveVolatile) { + COMPILE_ASSERT_TYPES_EQ(int, remove_volatile<int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_volatile<volatile int>::type); + COMPILE_ASSERT_TYPES_EQ(int *, remove_volatile<int * volatile>::type); + // TR1 examples. + COMPILE_ASSERT_TYPES_EQ(volatile int *, + remove_volatile<volatile int *>::type); + COMPILE_ASSERT_TYPES_EQ(const int, + remove_volatile<const volatile int>::type); +} + +TEST(TypeTraitsTest, TestRemoveCV) { + COMPILE_ASSERT_TYPES_EQ(int, remove_cv<int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_cv<volatile int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_cv<const int>::type); + COMPILE_ASSERT_TYPES_EQ(int *, remove_cv<int * const volatile>::type); + // TR1 examples. + COMPILE_ASSERT_TYPES_EQ(const volatile int *, + remove_cv<const volatile int *>::type); + COMPILE_ASSERT_TYPES_EQ(int, + remove_cv<const volatile int>::type); +} + +TEST(TypeTraitsTest, TestRemoveReference) { + COMPILE_ASSERT_TYPES_EQ(int, remove_reference<int>::type); + COMPILE_ASSERT_TYPES_EQ(int, remove_reference<int&>::type); + COMPILE_ASSERT_TYPES_EQ(const int, remove_reference<const int&>::type); + COMPILE_ASSERT_TYPES_EQ(int*, remove_reference<int * &>::type); +} + +TEST(TypeTraitsTest, TestIsSame) { + EXPECT_TRUE((is_same<int32, int32>::value)); + EXPECT_FALSE((is_same<int32, int64>::value)); + EXPECT_FALSE((is_same<int64, int32>::value)); + EXPECT_FALSE((is_same<int, const int>::value)); + + EXPECT_TRUE((is_same<void, void>::value)); + EXPECT_FALSE((is_same<void, int>::value)); + EXPECT_FALSE((is_same<int, void>::value)); + + EXPECT_TRUE((is_same<int*, int*>::value)); + EXPECT_TRUE((is_same<void*, void*>::value)); + EXPECT_FALSE((is_same<int*, void*>::value)); + EXPECT_FALSE((is_same<void*, int*>::value)); + EXPECT_FALSE((is_same<void*, const void*>::value)); + EXPECT_FALSE((is_same<void*, void* const>::value)); + + EXPECT_TRUE((is_same<Base*, Base*>::value)); + EXPECT_TRUE((is_same<Derived*, Derived*>::value)); + EXPECT_FALSE((is_same<Base*, Derived*>::value)); + EXPECT_FALSE((is_same<Derived*, Base*>::value)); +} + +TEST(TypeTraitsTest, TestConvertible) { +#if !defined(_MSC_VER) && !(defined(__GNUC__) && __GNUC__ <= 3) + EXPECT_TRUE((is_convertible<int, int>::value)); + EXPECT_TRUE((is_convertible<int, long>::value)); + EXPECT_TRUE((is_convertible<long, int>::value)); + + EXPECT_TRUE((is_convertible<int*, void*>::value)); + EXPECT_FALSE((is_convertible<void*, int*>::value)); + + EXPECT_TRUE((is_convertible<Derived*, Base*>::value)); + EXPECT_FALSE((is_convertible<Base*, Derived*>::value)); + EXPECT_TRUE((is_convertible<Derived*, const Base*>::value)); + EXPECT_FALSE((is_convertible<const Derived*, Base*>::value)); +#endif +} + +} // anonymous namespace +} // namespace protobuf +} // namespace google diff --git a/third_party/protobuf/src/google/protobuf/test_util.cc b/third_party/protobuf/src/google/protobuf/test_util.cc index af8b390..841f4b3 100644 --- a/third_party/protobuf/src/google/protobuf/test_util.cc +++ b/third_party/protobuf/src/google/protobuf/test_util.cc @@ -50,6 +50,13 @@ namespace google { namespace protobuf { void TestUtil::SetAllFields(unittest::TestAllTypes* message) { + SetOptionalFields(message); + AddRepeatedFields1(message); + AddRepeatedFields2(message); + SetDefaultFields(message); +} + +void TestUtil::SetOptionalFields(unittest::TestAllTypes* message) { message->set_optional_int32 (101); message->set_optional_int64 (102); message->set_optional_uint32 (103); @@ -66,10 +73,12 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->set_optional_string ("115"); message->set_optional_bytes ("116"); - message->mutable_optionalgroup ()->set_a(117); - message->mutable_optional_nested_message ()->set_bb(118); - message->mutable_optional_foreign_message()->set_c(119); - message->mutable_optional_import_message ()->set_d(120); + message->mutable_optionalgroup ()->set_a(117); + message->mutable_optional_nested_message ()->set_bb(118); + message->mutable_optional_foreign_message ()->set_c(119); + message->mutable_optional_import_message ()->set_d(120); + message->mutable_optional_public_import_message()->set_e(126); + message->mutable_optional_lazy_message ()->set_bb(127); message->set_optional_nested_enum (unittest::TestAllTypes::BAZ); message->set_optional_foreign_enum(unittest::FOREIGN_BAZ ); @@ -87,9 +96,11 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->GetDescriptor()->FindFieldByName("optional_cord"), "125"); #endif // !PROTOBUF_TEST_NO_DESCRIPTORS +} - // ----------------------------------------------------------------- +// ------------------------------------------------------------------- +void TestUtil::AddRepeatedFields1(unittest::TestAllTypes* message) { message->add_repeated_int32 (201); message->add_repeated_int64 (202); message->add_repeated_uint32 (203); @@ -110,6 +121,7 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->add_repeated_nested_message ()->set_bb(218); message->add_repeated_foreign_message()->set_c(219); message->add_repeated_import_message ()->set_d(220); + message->add_repeated_lazy_message ()->set_bb(227); message->add_repeated_nested_enum (unittest::TestAllTypes::BAR); message->add_repeated_foreign_enum(unittest::FOREIGN_BAR ); @@ -125,7 +137,9 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->GetDescriptor()->FindFieldByName("repeated_cord"), "225"); #endif // !PROTOBUF_TEST_NO_DESCRIPTORS +} +void TestUtil::AddRepeatedFields2(unittest::TestAllTypes* message) { // Add a second one of each field. message->add_repeated_int32 (301); message->add_repeated_int64 (302); @@ -147,6 +161,7 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->add_repeated_nested_message ()->set_bb(318); message->add_repeated_foreign_message()->set_c(319); message->add_repeated_import_message ()->set_d(320); + message->add_repeated_lazy_message ()->set_bb(327); message->add_repeated_nested_enum (unittest::TestAllTypes::BAZ); message->add_repeated_foreign_enum(unittest::FOREIGN_BAZ ); @@ -162,9 +177,11 @@ void TestUtil::SetAllFields(unittest::TestAllTypes* message) { message->GetDescriptor()->FindFieldByName("repeated_cord"), "325"); #endif // !PROTOBUF_TEST_NO_DESCRIPTORS +} - // ----------------------------------------------------------------- +// ------------------------------------------------------------------- +void TestUtil::SetDefaultFields(unittest::TestAllTypes* message) { message->set_default_int32 (401); message->set_default_int64 (402); message->set_default_uint32 (403); @@ -220,6 +237,7 @@ void TestUtil::ModifyRepeatedFields(unittest::TestAllTypes* message) { message->mutable_repeated_nested_message (1)->set_bb(518); message->mutable_repeated_foreign_message(1)->set_c(519); message->mutable_repeated_import_message (1)->set_d(520); + message->mutable_repeated_lazy_message (1)->set_bb(527); message->set_repeated_nested_enum (1, unittest::TestAllTypes::FOO); message->set_repeated_foreign_enum(1, unittest::FOREIGN_FOO ); @@ -256,15 +274,19 @@ void TestUtil::ExpectAllFieldsSet(const unittest::TestAllTypes& message) { EXPECT_TRUE(message.has_optional_string ()); EXPECT_TRUE(message.has_optional_bytes ()); - EXPECT_TRUE(message.has_optionalgroup ()); - EXPECT_TRUE(message.has_optional_nested_message ()); - EXPECT_TRUE(message.has_optional_foreign_message()); - EXPECT_TRUE(message.has_optional_import_message ()); + EXPECT_TRUE(message.has_optionalgroup ()); + EXPECT_TRUE(message.has_optional_nested_message ()); + EXPECT_TRUE(message.has_optional_foreign_message ()); + EXPECT_TRUE(message.has_optional_import_message ()); + EXPECT_TRUE(message.has_optional_public_import_message()); + EXPECT_TRUE(message.has_optional_lazy_message ()); - EXPECT_TRUE(message.optionalgroup ().has_a()); - EXPECT_TRUE(message.optional_nested_message ().has_bb()); - EXPECT_TRUE(message.optional_foreign_message().has_c()); - EXPECT_TRUE(message.optional_import_message ().has_d()); + EXPECT_TRUE(message.optionalgroup ().has_a()); + EXPECT_TRUE(message.optional_nested_message ().has_bb()); + EXPECT_TRUE(message.optional_foreign_message ().has_c()); + EXPECT_TRUE(message.optional_import_message ().has_d()); + EXPECT_TRUE(message.optional_public_import_message().has_e()); + EXPECT_TRUE(message.optional_lazy_message ().has_bb()); EXPECT_TRUE(message.has_optional_nested_enum ()); EXPECT_TRUE(message.has_optional_foreign_enum()); @@ -291,10 +313,12 @@ void TestUtil::ExpectAllFieldsSet(const unittest::TestAllTypes& message) { EXPECT_EQ("115", message.optional_string ()); EXPECT_EQ("116", message.optional_bytes ()); - EXPECT_EQ(117, message.optionalgroup ().a()); - EXPECT_EQ(118, message.optional_nested_message ().bb()); - EXPECT_EQ(119, message.optional_foreign_message().c()); - EXPECT_EQ(120, message.optional_import_message ().d()); + EXPECT_EQ(117, message.optionalgroup ().a()); + EXPECT_EQ(118, message.optional_nested_message ().bb()); + EXPECT_EQ(119, message.optional_foreign_message ().c()); + EXPECT_EQ(120, message.optional_import_message ().d()); + EXPECT_EQ(126, message.optional_public_import_message ().e()); + EXPECT_EQ(127, message.optional_lazy_message ().bb()); EXPECT_EQ(unittest::TestAllTypes::BAZ, message.optional_nested_enum ()); EXPECT_EQ(unittest::FOREIGN_BAZ , message.optional_foreign_enum()); @@ -323,6 +347,7 @@ void TestUtil::ExpectAllFieldsSet(const unittest::TestAllTypes& message) { ASSERT_EQ(2, message.repeated_nested_message_size ()); ASSERT_EQ(2, message.repeated_foreign_message_size()); ASSERT_EQ(2, message.repeated_import_message_size ()); + ASSERT_EQ(2, message.repeated_lazy_message_size ()); ASSERT_EQ(2, message.repeated_nested_enum_size ()); ASSERT_EQ(2, message.repeated_foreign_enum_size ()); ASSERT_EQ(2, message.repeated_import_enum_size ()); @@ -352,6 +377,7 @@ void TestUtil::ExpectAllFieldsSet(const unittest::TestAllTypes& message) { EXPECT_EQ(218, message.repeated_nested_message (0).bb()); EXPECT_EQ(219, message.repeated_foreign_message(0).c()); EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(227, message.repeated_lazy_message (0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.repeated_nested_enum (0)); @@ -378,6 +404,7 @@ void TestUtil::ExpectAllFieldsSet(const unittest::TestAllTypes& message) { EXPECT_EQ(318, message.repeated_nested_message (1).bb()); EXPECT_EQ(319, message.repeated_foreign_message(1).c()); EXPECT_EQ(320, message.repeated_import_message (1).d()); + EXPECT_EQ(327, message.repeated_lazy_message (1).bb()); EXPECT_EQ(unittest::TestAllTypes::BAZ, message.repeated_nested_enum (1)); EXPECT_EQ(unittest::FOREIGN_BAZ , message.repeated_foreign_enum(1)); @@ -449,10 +476,12 @@ void TestUtil::ExpectClear(const unittest::TestAllTypes& message) { EXPECT_FALSE(message.has_optional_string ()); EXPECT_FALSE(message.has_optional_bytes ()); - EXPECT_FALSE(message.has_optionalgroup ()); - EXPECT_FALSE(message.has_optional_nested_message ()); - EXPECT_FALSE(message.has_optional_foreign_message()); - EXPECT_FALSE(message.has_optional_import_message ()); + EXPECT_FALSE(message.has_optionalgroup ()); + EXPECT_FALSE(message.has_optional_nested_message ()); + EXPECT_FALSE(message.has_optional_foreign_message ()); + EXPECT_FALSE(message.has_optional_import_message ()); + EXPECT_FALSE(message.has_optional_public_import_message()); + EXPECT_FALSE(message.has_optional_lazy_message ()); EXPECT_FALSE(message.has_optional_nested_enum ()); EXPECT_FALSE(message.has_optional_foreign_enum()); @@ -479,15 +508,19 @@ void TestUtil::ExpectClear(const unittest::TestAllTypes& message) { EXPECT_EQ("" , message.optional_bytes ()); // Embedded messages should also be clear. - EXPECT_FALSE(message.optionalgroup ().has_a()); - EXPECT_FALSE(message.optional_nested_message ().has_bb()); - EXPECT_FALSE(message.optional_foreign_message().has_c()); - EXPECT_FALSE(message.optional_import_message ().has_d()); - - EXPECT_EQ(0, message.optionalgroup ().a()); - EXPECT_EQ(0, message.optional_nested_message ().bb()); - EXPECT_EQ(0, message.optional_foreign_message().c()); - EXPECT_EQ(0, message.optional_import_message ().d()); + EXPECT_FALSE(message.optionalgroup ().has_a()); + EXPECT_FALSE(message.optional_nested_message ().has_bb()); + EXPECT_FALSE(message.optional_foreign_message ().has_c()); + EXPECT_FALSE(message.optional_import_message ().has_d()); + EXPECT_FALSE(message.optional_public_import_message().has_e()); + EXPECT_FALSE(message.optional_lazy_message ().has_bb()); + + EXPECT_EQ(0, message.optionalgroup ().a()); + EXPECT_EQ(0, message.optional_nested_message ().bb()); + EXPECT_EQ(0, message.optional_foreign_message ().c()); + EXPECT_EQ(0, message.optional_import_message ().d()); + EXPECT_EQ(0, message.optional_public_import_message().e()); + EXPECT_EQ(0, message.optional_lazy_message ().bb()); // Enums without defaults are set to the first value in the enum. EXPECT_EQ(unittest::TestAllTypes::FOO, message.optional_nested_enum ()); @@ -516,6 +549,7 @@ void TestUtil::ExpectClear(const unittest::TestAllTypes& message) { EXPECT_EQ(0, message.repeated_nested_message_size ()); EXPECT_EQ(0, message.repeated_foreign_message_size()); EXPECT_EQ(0, message.repeated_import_message_size ()); + EXPECT_EQ(0, message.repeated_lazy_message_size ()); EXPECT_EQ(0, message.repeated_nested_enum_size ()); EXPECT_EQ(0, message.repeated_foreign_enum_size ()); EXPECT_EQ(0, message.repeated_import_enum_size ()); @@ -595,6 +629,7 @@ void TestUtil::ExpectRepeatedFieldsModified( ASSERT_EQ(2, message.repeated_nested_message_size ()); ASSERT_EQ(2, message.repeated_foreign_message_size()); ASSERT_EQ(2, message.repeated_import_message_size ()); + ASSERT_EQ(2, message.repeated_lazy_message_size ()); ASSERT_EQ(2, message.repeated_nested_enum_size ()); ASSERT_EQ(2, message.repeated_foreign_enum_size ()); ASSERT_EQ(2, message.repeated_import_enum_size ()); @@ -624,6 +659,7 @@ void TestUtil::ExpectRepeatedFieldsModified( EXPECT_EQ(218, message.repeated_nested_message (0).bb()); EXPECT_EQ(219, message.repeated_foreign_message(0).c()); EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(227, message.repeated_lazy_message (0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.repeated_nested_enum (0)); EXPECT_EQ(unittest::FOREIGN_BAR , message.repeated_foreign_enum(0)); @@ -651,6 +687,7 @@ void TestUtil::ExpectRepeatedFieldsModified( EXPECT_EQ(518, message.repeated_nested_message (1).bb()); EXPECT_EQ(519, message.repeated_foreign_message(1).c()); EXPECT_EQ(520, message.repeated_import_message (1).d()); + EXPECT_EQ(527, message.repeated_lazy_message (1).bb()); EXPECT_EQ(unittest::TestAllTypes::FOO, message.repeated_nested_enum (1)); EXPECT_EQ(unittest::FOREIGN_FOO , message.repeated_foreign_enum(1)); @@ -953,6 +990,9 @@ void TestUtil::SetAllExtensions(unittest::TestAllExtensions* message) { message->SetExtension(unittest::optional_string_piece_extension, "124"); message->SetExtension(unittest::optional_cord_extension, "125"); + message->MutableExtension(unittest::optional_public_import_message_extension)->set_e(126); + message->MutableExtension(unittest::optional_lazy_message_extension)->set_bb(127); + // ----------------------------------------------------------------- message->AddExtension(unittest::repeated_int32_extension , 201); @@ -975,6 +1015,7 @@ void TestUtil::SetAllExtensions(unittest::TestAllExtensions* message) { message->AddExtension(unittest::repeated_nested_message_extension )->set_bb(218); message->AddExtension(unittest::repeated_foreign_message_extension)->set_c(219); message->AddExtension(unittest::repeated_import_message_extension )->set_d(220); + message->AddExtension(unittest::repeated_lazy_message_extension )->set_bb(227); message->AddExtension(unittest::repeated_nested_enum_extension , unittest::TestAllTypes::BAR); message->AddExtension(unittest::repeated_foreign_enum_extension, unittest::FOREIGN_BAR ); @@ -1004,6 +1045,7 @@ void TestUtil::SetAllExtensions(unittest::TestAllExtensions* message) { message->AddExtension(unittest::repeated_nested_message_extension )->set_bb(318); message->AddExtension(unittest::repeated_foreign_message_extension)->set_c(319); message->AddExtension(unittest::repeated_import_message_extension )->set_d(320); + message->AddExtension(unittest::repeated_lazy_message_extension )->set_bb(327); message->AddExtension(unittest::repeated_nested_enum_extension , unittest::TestAllTypes::BAZ); message->AddExtension(unittest::repeated_foreign_enum_extension, unittest::FOREIGN_BAZ ); @@ -1073,6 +1115,7 @@ void TestUtil::ModifyRepeatedExtensions(unittest::TestAllExtensions* message) { message->MutableExtension(unittest::repeated_nested_message_extension , 1)->set_bb(518); message->MutableExtension(unittest::repeated_foreign_message_extension, 1)->set_c(519); message->MutableExtension(unittest::repeated_import_message_extension , 1)->set_d(520); + message->MutableExtension(unittest::repeated_lazy_message_extension , 1)->set_bb(527); message->SetExtension(unittest::repeated_nested_enum_extension , 1, unittest::TestAllTypes::FOO); message->SetExtension(unittest::repeated_foreign_enum_extension, 1, unittest::FOREIGN_FOO ); @@ -1102,15 +1145,19 @@ void TestUtil::ExpectAllExtensionsSet( EXPECT_TRUE(message.HasExtension(unittest::optional_string_extension )); EXPECT_TRUE(message.HasExtension(unittest::optional_bytes_extension )); - EXPECT_TRUE(message.HasExtension(unittest::optionalgroup_extension )); - EXPECT_TRUE(message.HasExtension(unittest::optional_nested_message_extension )); - EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_message_extension)); - EXPECT_TRUE(message.HasExtension(unittest::optional_import_message_extension )); + EXPECT_TRUE(message.HasExtension(unittest::optionalgroup_extension )); + EXPECT_TRUE(message.HasExtension(unittest::optional_nested_message_extension )); + EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_message_extension )); + EXPECT_TRUE(message.HasExtension(unittest::optional_import_message_extension )); + EXPECT_TRUE(message.HasExtension(unittest::optional_public_import_message_extension)); + EXPECT_TRUE(message.HasExtension(unittest::optional_lazy_message_extension )); - EXPECT_TRUE(message.GetExtension(unittest::optionalgroup_extension ).has_a()); - EXPECT_TRUE(message.GetExtension(unittest::optional_nested_message_extension ).has_bb()); - EXPECT_TRUE(message.GetExtension(unittest::optional_foreign_message_extension).has_c()); - EXPECT_TRUE(message.GetExtension(unittest::optional_import_message_extension ).has_d()); + EXPECT_TRUE(message.GetExtension(unittest::optionalgroup_extension ).has_a()); + EXPECT_TRUE(message.GetExtension(unittest::optional_nested_message_extension ).has_bb()); + EXPECT_TRUE(message.GetExtension(unittest::optional_foreign_message_extension ).has_c()); + EXPECT_TRUE(message.GetExtension(unittest::optional_import_message_extension ).has_d()); + EXPECT_TRUE(message.GetExtension(unittest::optional_public_import_message_extension).has_e()); + EXPECT_TRUE(message.GetExtension(unittest::optional_lazy_message_extension ).has_bb()); EXPECT_TRUE(message.HasExtension(unittest::optional_nested_enum_extension )); EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_enum_extension)); @@ -1146,6 +1193,8 @@ void TestUtil::ExpectAllExtensionsSet( EXPECT_EQ("124", message.GetExtension(unittest::optional_string_piece_extension)); EXPECT_EQ("125", message.GetExtension(unittest::optional_cord_extension)); + EXPECT_EQ(126, message.GetExtension(unittest::optional_public_import_message_extension ).e()); + EXPECT_EQ(127, message.GetExtension(unittest::optional_lazy_message_extension).bb()); // ----------------------------------------------------------------- @@ -1169,6 +1218,7 @@ void TestUtil::ExpectAllExtensionsSet( ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_message_extension)); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_message_extension )); + ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_lazy_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_enum_extension )); @@ -1196,6 +1246,7 @@ void TestUtil::ExpectAllExtensionsSet( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension , 0).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension, 0).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension , 0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.GetExtension(unittest::repeated_nested_enum_extension , 0)); EXPECT_EQ(unittest::FOREIGN_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension, 0)); @@ -1224,6 +1275,7 @@ void TestUtil::ExpectAllExtensionsSet( EXPECT_EQ(318, message.GetExtension(unittest::repeated_nested_message_extension , 1).bb()); EXPECT_EQ(319, message.GetExtension(unittest::repeated_foreign_message_extension, 1).c()); EXPECT_EQ(320, message.GetExtension(unittest::repeated_import_message_extension , 1).d()); + EXPECT_EQ(327, message.GetExtension(unittest::repeated_lazy_message_extension , 1).bb()); EXPECT_EQ(unittest::TestAllTypes::BAZ, message.GetExtension(unittest::repeated_nested_enum_extension , 1)); EXPECT_EQ(unittest::FOREIGN_BAZ , message.GetExtension(unittest::repeated_foreign_enum_extension, 1)); @@ -1307,10 +1359,12 @@ void TestUtil::ExpectExtensionsClear( EXPECT_FALSE(message.HasExtension(unittest::optional_string_extension )); EXPECT_FALSE(message.HasExtension(unittest::optional_bytes_extension )); - EXPECT_FALSE(message.HasExtension(unittest::optionalgroup_extension )); - EXPECT_FALSE(message.HasExtension(unittest::optional_nested_message_extension )); - EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_message_extension)); - EXPECT_FALSE(message.HasExtension(unittest::optional_import_message_extension )); + EXPECT_FALSE(message.HasExtension(unittest::optionalgroup_extension )); + EXPECT_FALSE(message.HasExtension(unittest::optional_nested_message_extension )); + EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_message_extension )); + EXPECT_FALSE(message.HasExtension(unittest::optional_import_message_extension )); + EXPECT_FALSE(message.HasExtension(unittest::optional_public_import_message_extension)); + EXPECT_FALSE(message.HasExtension(unittest::optional_lazy_message_extension )); EXPECT_FALSE(message.HasExtension(unittest::optional_nested_enum_extension )); EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_enum_extension)); @@ -1337,15 +1391,19 @@ void TestUtil::ExpectExtensionsClear( EXPECT_EQ("" , message.GetExtension(unittest::optional_bytes_extension )); // Embedded messages should also be clear. - EXPECT_FALSE(message.GetExtension(unittest::optionalgroup_extension ).has_a()); - EXPECT_FALSE(message.GetExtension(unittest::optional_nested_message_extension ).has_bb()); - EXPECT_FALSE(message.GetExtension(unittest::optional_foreign_message_extension).has_c()); - EXPECT_FALSE(message.GetExtension(unittest::optional_import_message_extension ).has_d()); - - EXPECT_EQ(0, message.GetExtension(unittest::optionalgroup_extension ).a()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_nested_message_extension ).bb()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_foreign_message_extension).c()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_import_message_extension ).d()); + EXPECT_FALSE(message.GetExtension(unittest::optionalgroup_extension ).has_a()); + EXPECT_FALSE(message.GetExtension(unittest::optional_nested_message_extension ).has_bb()); + EXPECT_FALSE(message.GetExtension(unittest::optional_foreign_message_extension ).has_c()); + EXPECT_FALSE(message.GetExtension(unittest::optional_import_message_extension ).has_d()); + EXPECT_FALSE(message.GetExtension(unittest::optional_public_import_message_extension).has_e()); + EXPECT_FALSE(message.GetExtension(unittest::optional_lazy_message_extension ).has_bb()); + + EXPECT_EQ(0, message.GetExtension(unittest::optionalgroup_extension ).a()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_nested_message_extension ).bb()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_foreign_message_extension ).c()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_import_message_extension ).d()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_public_import_message_extension).e()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_lazy_message_extension ).bb()); // Enums without defaults are set to the first value in the enum. EXPECT_EQ(unittest::TestAllTypes::FOO, message.GetExtension(unittest::optional_nested_enum_extension )); @@ -1376,6 +1434,7 @@ void TestUtil::ExpectExtensionsClear( EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_nested_message_extension )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_foreign_message_extension)); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_import_message_extension )); + EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_lazy_message_extension )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_nested_enum_extension )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_foreign_enum_extension )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_import_enum_extension )); @@ -1459,6 +1518,7 @@ void TestUtil::ExpectRepeatedExtensionsModified( ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_message_extension)); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_message_extension )); + ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_lazy_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_enum_extension )); @@ -1486,6 +1546,7 @@ void TestUtil::ExpectRepeatedExtensionsModified( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension , 0).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension, 0).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension , 0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.GetExtension(unittest::repeated_nested_enum_extension , 0)); EXPECT_EQ(unittest::FOREIGN_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension, 0)); @@ -1515,6 +1576,7 @@ void TestUtil::ExpectRepeatedExtensionsModified( EXPECT_EQ(518, message.GetExtension(unittest::repeated_nested_message_extension , 1).bb()); EXPECT_EQ(519, message.GetExtension(unittest::repeated_foreign_message_extension, 1).c()); EXPECT_EQ(520, message.GetExtension(unittest::repeated_import_message_extension , 1).d()); + EXPECT_EQ(527, message.GetExtension(unittest::repeated_lazy_message_extension , 1).bb()); EXPECT_EQ(unittest::TestAllTypes::FOO, message.GetExtension(unittest::repeated_nested_enum_extension , 1)); EXPECT_EQ(unittest::FOREIGN_FOO , message.GetExtension(unittest::repeated_foreign_enum_extension, 1)); @@ -1750,6 +1812,7 @@ void TestUtil::ExpectLastRepeatedsRemoved( ASSERT_EQ(1, message.repeated_nested_message_size ()); ASSERT_EQ(1, message.repeated_foreign_message_size()); ASSERT_EQ(1, message.repeated_import_message_size ()); + ASSERT_EQ(1, message.repeated_import_message_size ()); ASSERT_EQ(1, message.repeated_nested_enum_size ()); ASSERT_EQ(1, message.repeated_foreign_enum_size ()); ASSERT_EQ(1, message.repeated_import_enum_size ()); @@ -1780,6 +1843,7 @@ void TestUtil::ExpectLastRepeatedsRemoved( EXPECT_EQ(218, message.repeated_nested_message (0).bb()); EXPECT_EQ(219, message.repeated_foreign_message(0).c()); EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(220, message.repeated_import_message (0).d()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.repeated_nested_enum (0)); EXPECT_EQ(unittest::FOREIGN_BAR , message.repeated_foreign_enum(0)); @@ -1810,6 +1874,7 @@ void TestUtil::ExpectLastRepeatedExtensionsRemoved( ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_nested_message_extension )); ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_foreign_message_extension)); ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_import_message_extension )); + ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_lazy_message_extension )); ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_nested_enum_extension )); ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_foreign_enum_extension )); ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_import_enum_extension )); @@ -1838,6 +1903,7 @@ void TestUtil::ExpectLastRepeatedExtensionsRemoved( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension , 0).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension, 0).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension , 0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.GetExtension(unittest::repeated_nested_enum_extension , 0)); EXPECT_EQ(unittest::FOREIGN_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension, 0)); @@ -1847,6 +1913,36 @@ void TestUtil::ExpectLastRepeatedExtensionsRemoved( EXPECT_EQ("225", message.GetExtension(unittest::repeated_cord_extension, 0)); } +void TestUtil::ExpectLastRepeatedsReleased( + const unittest::TestAllTypes& message) { + ASSERT_EQ(1, message.repeatedgroup_size ()); + ASSERT_EQ(1, message.repeated_nested_message_size ()); + ASSERT_EQ(1, message.repeated_foreign_message_size()); + ASSERT_EQ(1, message.repeated_import_message_size ()); + ASSERT_EQ(1, message.repeated_import_message_size ()); + + EXPECT_EQ(217, message.repeatedgroup (0).a()); + EXPECT_EQ(218, message.repeated_nested_message (0).bb()); + EXPECT_EQ(219, message.repeated_foreign_message(0).c()); + EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(220, message.repeated_import_message (0).d()); +} + +void TestUtil::ExpectLastRepeatedExtensionsReleased( + const unittest::TestAllExtensions& message) { + ASSERT_EQ(1, message.ExtensionSize(unittest::repeatedgroup_extension )); + ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_nested_message_extension )); + ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_foreign_message_extension)); + ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_import_message_extension )); + ASSERT_EQ(1, message.ExtensionSize(unittest::repeated_lazy_message_extension )); + + EXPECT_EQ(217, message.GetExtension(unittest::repeatedgroup_extension , 0).a()); + EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension , 0).bb()); + EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension, 0).c()); + EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension , 0).bb()); +} + void TestUtil::ExpectRepeatedsSwapped( const unittest::TestAllTypes& message) { ASSERT_EQ(2, message.repeated_int32_size ()); @@ -1869,6 +1965,7 @@ void TestUtil::ExpectRepeatedsSwapped( ASSERT_EQ(2, message.repeated_nested_message_size ()); ASSERT_EQ(2, message.repeated_foreign_message_size()); ASSERT_EQ(2, message.repeated_import_message_size ()); + ASSERT_EQ(2, message.repeated_import_message_size ()); ASSERT_EQ(2, message.repeated_nested_enum_size ()); ASSERT_EQ(2, message.repeated_foreign_enum_size ()); ASSERT_EQ(2, message.repeated_import_enum_size ()); @@ -1899,6 +1996,7 @@ void TestUtil::ExpectRepeatedsSwapped( EXPECT_EQ(218, message.repeated_nested_message (1).bb()); EXPECT_EQ(219, message.repeated_foreign_message(1).c()); EXPECT_EQ(220, message.repeated_import_message (1).d()); + EXPECT_EQ(220, message.repeated_import_message (1).d()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.repeated_nested_enum (1)); EXPECT_EQ(unittest::FOREIGN_BAR , message.repeated_foreign_enum(1)); @@ -1924,6 +2022,7 @@ void TestUtil::ExpectRepeatedsSwapped( EXPECT_EQ(318, message.repeated_nested_message (0).bb()); EXPECT_EQ(319, message.repeated_foreign_message(0).c()); EXPECT_EQ(320, message.repeated_import_message (0).d()); + EXPECT_EQ(320, message.repeated_import_message (0).d()); EXPECT_EQ(unittest::TestAllTypes::BAZ, message.repeated_nested_enum (0)); EXPECT_EQ(unittest::FOREIGN_BAZ , message.repeated_foreign_enum(0)); @@ -1953,6 +2052,7 @@ void TestUtil::ExpectRepeatedExtensionsSwapped( ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_message_extension)); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_message_extension )); + ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_lazy_message_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_enum_extension )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_enum_extension )); @@ -1980,6 +2080,7 @@ void TestUtil::ExpectRepeatedExtensionsSwapped( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension , 1).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension, 1).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension , 1).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension , 1).bb()); EXPECT_EQ(unittest::TestAllTypes::BAR, message.GetExtension(unittest::repeated_nested_enum_extension , 1)); EXPECT_EQ(unittest::FOREIGN_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension, 1)); @@ -2008,6 +2109,7 @@ void TestUtil::ExpectRepeatedExtensionsSwapped( EXPECT_EQ(318, message.GetExtension(unittest::repeated_nested_message_extension , 0).bb()); EXPECT_EQ(319, message.GetExtension(unittest::repeated_foreign_message_extension, 0).c()); EXPECT_EQ(320, message.GetExtension(unittest::repeated_import_message_extension , 0).d()); + EXPECT_EQ(327, message.GetExtension(unittest::repeated_lazy_message_extension , 0).bb()); EXPECT_EQ(unittest::TestAllTypes::BAZ, message.GetExtension(unittest::repeated_nested_enum_extension , 0)); EXPECT_EQ(unittest::FOREIGN_BAZ , message.GetExtension(unittest::repeated_foreign_enum_extension, 0)); @@ -2031,6 +2133,8 @@ TestUtil::ReflectionTester::ReflectionTester( pool->FindFieldByName("protobuf_unittest.ForeignMessage.c"); import_d_ = pool->FindFieldByName("protobuf_unittest_import.ImportMessage.d"); + import_e_ = + pool->FindFieldByName("protobuf_unittest_import.PublicImportMessage.e"); nested_foo_ = pool->FindEnumValueByName("protobuf_unittest.TestAllTypes.FOO"); nested_bar_ = @@ -2067,6 +2171,7 @@ TestUtil::ReflectionTester::ReflectionTester( EXPECT_TRUE(nested_b_ != NULL); EXPECT_TRUE(foreign_c_ != NULL); EXPECT_TRUE(import_d_ != NULL); + EXPECT_TRUE(import_e_ != NULL); EXPECT_TRUE(nested_foo_ != NULL); EXPECT_TRUE(nested_bar_ != NULL); EXPECT_TRUE(nested_baz_ != NULL); @@ -2129,6 +2234,12 @@ void TestUtil::ReflectionTester::SetAllFieldsViaReflection(Message* message) { reflection->SetString(message, F("optional_string_piece"), "124"); reflection->SetString(message, F("optional_cord"), "125"); + sub_message = reflection->MutableMessage(message, F("optional_public_import_message")); + sub_message->GetReflection()->SetInt32(sub_message, import_e_, 126); + + sub_message = reflection->MutableMessage(message, F("optional_lazy_message")); + sub_message->GetReflection()->SetInt32(sub_message, nested_b_, 127); + // ----------------------------------------------------------------- reflection->AddInt32 (message, F("repeated_int32" ), 201); @@ -2155,6 +2266,8 @@ void TestUtil::ReflectionTester::SetAllFieldsViaReflection(Message* message) { sub_message->GetReflection()->SetInt32(sub_message, foreign_c_, 219); sub_message = reflection->AddMessage(message, F("repeated_import_message")); sub_message->GetReflection()->SetInt32(sub_message, import_d_, 220); + sub_message = reflection->AddMessage(message, F("repeated_lazy_message")); + sub_message->GetReflection()->SetInt32(sub_message, nested_b_, 227); reflection->AddEnum(message, F("repeated_nested_enum" ), nested_bar_); reflection->AddEnum(message, F("repeated_foreign_enum"), foreign_bar_); @@ -2188,6 +2301,8 @@ void TestUtil::ReflectionTester::SetAllFieldsViaReflection(Message* message) { sub_message->GetReflection()->SetInt32(sub_message, foreign_c_, 319); sub_message = reflection->AddMessage(message, F("repeated_import_message")); sub_message->GetReflection()->SetInt32(sub_message, import_d_, 320); + sub_message = reflection->AddMessage(message, F("repeated_lazy_message")); + sub_message->GetReflection()->SetInt32(sub_message, nested_b_, 327); reflection->AddEnum(message, F("repeated_nested_enum" ), nested_baz_); reflection->AddEnum(message, F("repeated_foreign_enum"), foreign_baz_); @@ -2289,10 +2404,12 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection1( EXPECT_TRUE(reflection->HasField(message, F("optional_string" ))); EXPECT_TRUE(reflection->HasField(message, F("optional_bytes" ))); - EXPECT_TRUE(reflection->HasField(message, F("optionalgroup" ))); - EXPECT_TRUE(reflection->HasField(message, F("optional_nested_message" ))); - EXPECT_TRUE(reflection->HasField(message, F("optional_foreign_message"))); - EXPECT_TRUE(reflection->HasField(message, F("optional_import_message" ))); + EXPECT_TRUE(reflection->HasField(message, F("optionalgroup" ))); + EXPECT_TRUE(reflection->HasField(message, F("optional_nested_message" ))); + EXPECT_TRUE(reflection->HasField(message, F("optional_foreign_message" ))); + EXPECT_TRUE(reflection->HasField(message, F("optional_import_message" ))); + EXPECT_TRUE(reflection->HasField(message, F("optional_public_import_message"))); + EXPECT_TRUE(reflection->HasField(message, F("optional_lazy_message" ))); sub_message = &reflection->GetMessage(message, F("optionalgroup")); EXPECT_TRUE(sub_message->GetReflection()->HasField(*sub_message, group_a_)); @@ -2302,6 +2419,10 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection1( EXPECT_TRUE(sub_message->GetReflection()->HasField(*sub_message, foreign_c_)); sub_message = &reflection->GetMessage(message, F("optional_import_message")); EXPECT_TRUE(sub_message->GetReflection()->HasField(*sub_message, import_d_)); + sub_message = &reflection->GetMessage(message, F("optional_public_import_message")); + EXPECT_TRUE(sub_message->GetReflection()->HasField(*sub_message, import_e_)); + sub_message = &reflection->GetMessage(message, F("optional_lazy_message")); + EXPECT_TRUE(sub_message->GetReflection()->HasField(*sub_message, nested_b_)); EXPECT_TRUE(reflection->HasField(message, F("optional_nested_enum" ))); EXPECT_TRUE(reflection->HasField(message, F("optional_foreign_enum"))); @@ -2337,6 +2458,10 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection1( EXPECT_EQ(119, sub_message->GetReflection()->GetInt32(*sub_message, foreign_c_)); sub_message = &reflection->GetMessage(message, F("optional_import_message")); EXPECT_EQ(120, sub_message->GetReflection()->GetInt32(*sub_message, import_d_)); + sub_message = &reflection->GetMessage(message, F("optional_public_import_message")); + EXPECT_EQ(126, sub_message->GetReflection()->GetInt32(*sub_message, import_e_)); + sub_message = &reflection->GetMessage(message, F("optional_lazy_message")); + EXPECT_EQ(127, sub_message->GetReflection()->GetInt32(*sub_message, nested_b_)); EXPECT_EQ( nested_baz_, reflection->GetEnum(message, F("optional_nested_enum" ))); EXPECT_EQ(foreign_baz_, reflection->GetEnum(message, F("optional_foreign_enum"))); @@ -2347,6 +2472,7 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection1( EXPECT_EQ("125", reflection->GetString(message, F("optional_cord"))); EXPECT_EQ("125", reflection->GetStringReference(message, F("optional_cord"), &scratch)); + } void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection2( @@ -2377,6 +2503,7 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection2( ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_nested_message" ))); ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_foreign_message"))); ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_import_message" ))); + ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_lazy_message" ))); ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_nested_enum" ))); ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_foreign_enum" ))); ASSERT_EQ(2, reflection->FieldSize(message, F("repeated_import_enum" ))); @@ -2411,6 +2538,8 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection2( EXPECT_EQ(219, sub_message->GetReflection()->GetInt32(*sub_message, foreign_c_)); sub_message = &reflection->GetRepeatedMessage(message, F("repeated_import_message"), 0); EXPECT_EQ(220, sub_message->GetReflection()->GetInt32(*sub_message, import_d_)); + sub_message = &reflection->GetRepeatedMessage(message, F("repeated_lazy_message"), 0); + EXPECT_EQ(227, sub_message->GetReflection()->GetInt32(*sub_message, nested_b_)); EXPECT_EQ( nested_bar_, reflection->GetRepeatedEnum(message, F("repeated_nested_enum" ),0)); EXPECT_EQ(foreign_bar_, reflection->GetRepeatedEnum(message, F("repeated_foreign_enum"),0)); @@ -2453,6 +2582,8 @@ void TestUtil::ReflectionTester::ExpectAllFieldsSetViaReflection2( EXPECT_EQ(319, sub_message->GetReflection()->GetInt32(*sub_message, foreign_c_)); sub_message = &reflection->GetRepeatedMessage(message, F("repeated_import_message"), 1); EXPECT_EQ(320, sub_message->GetReflection()->GetInt32(*sub_message, import_d_)); + sub_message = &reflection->GetRepeatedMessage(message, F("repeated_lazy_message"), 1); + EXPECT_EQ(327, sub_message->GetReflection()->GetInt32(*sub_message, nested_b_)); EXPECT_EQ( nested_baz_, reflection->GetRepeatedEnum(message, F("repeated_nested_enum" ),1)); EXPECT_EQ(foreign_baz_, reflection->GetRepeatedEnum(message, F("repeated_foreign_enum"),1)); @@ -2609,6 +2740,8 @@ void TestUtil::ReflectionTester::ExpectClearViaReflection( EXPECT_FALSE(reflection->HasField(message, F("optional_nested_message" ))); EXPECT_FALSE(reflection->HasField(message, F("optional_foreign_message"))); EXPECT_FALSE(reflection->HasField(message, F("optional_import_message" ))); + EXPECT_FALSE(reflection->HasField(message, F("optional_public_import_message"))); + EXPECT_FALSE(reflection->HasField(message, F("optional_lazy_message"))); EXPECT_FALSE(reflection->HasField(message, F("optional_nested_enum" ))); EXPECT_FALSE(reflection->HasField(message, F("optional_foreign_enum"))); @@ -2650,6 +2783,12 @@ void TestUtil::ReflectionTester::ExpectClearViaReflection( sub_message = &reflection->GetMessage(message, F("optional_import_message")); EXPECT_FALSE(sub_message->GetReflection()->HasField(*sub_message, import_d_)); EXPECT_EQ(0, sub_message->GetReflection()->GetInt32(*sub_message, import_d_)); + sub_message = &reflection->GetMessage(message, F("optional_public_import_message")); + EXPECT_FALSE(sub_message->GetReflection()->HasField(*sub_message, import_e_)); + EXPECT_EQ(0, sub_message->GetReflection()->GetInt32(*sub_message, import_e_)); + sub_message = &reflection->GetMessage(message, F("optional_lazy_message")); + EXPECT_FALSE(sub_message->GetReflection()->HasField(*sub_message, nested_b_)); + EXPECT_EQ(0, sub_message->GetReflection()->GetInt32(*sub_message, nested_b_)); // Enums without defaults are set to the first value in the enum. EXPECT_EQ( nested_foo_, reflection->GetEnum(message, F("optional_nested_enum" ))); @@ -2683,6 +2822,7 @@ void TestUtil::ReflectionTester::ExpectClearViaReflection( EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_nested_message" ))); EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_foreign_message"))); EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_import_message" ))); + EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_lazy_message" ))); EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_nested_enum" ))); EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_foreign_enum" ))); EXPECT_EQ(0, reflection->FieldSize(message, F("repeated_import_enum" ))); @@ -2796,6 +2936,8 @@ void TestUtil::ReflectionTester::ModifyRepeatedFieldsViaReflection( sub_message->GetReflection()->SetInt32(sub_message, foreign_c_, 519); sub_message = reflection->MutableRepeatedMessage(message, F("repeated_import_message"), 1); sub_message->GetReflection()->SetInt32(sub_message, import_d_, 520); + sub_message = reflection->MutableRepeatedMessage(message, F("repeated_lazy_message"), 1); + sub_message->GetReflection()->SetInt32(sub_message, nested_b_, 527); reflection->SetRepeatedEnum(message, F("repeated_nested_enum" ), 1, nested_foo_); reflection->SetRepeatedEnum(message, F("repeated_foreign_enum"), 1, foreign_foo_); @@ -2824,7 +2966,8 @@ void TestUtil::ReflectionTester::ModifyPackedFieldsViaReflection( reflection->SetRepeatedEnum (message, F("packed_enum" ), 1, foreign_foo_); } -void TestUtil::ReflectionTester::RemoveLastRepeatedsViaReflection(Message* message) { +void TestUtil::ReflectionTester::RemoveLastRepeatedsViaReflection( + Message* message) { const Reflection* reflection = message->GetReflection(); vector<const FieldDescriptor*> output; @@ -2837,6 +2980,26 @@ void TestUtil::ReflectionTester::RemoveLastRepeatedsViaReflection(Message* messa } } +void TestUtil::ReflectionTester::ReleaseLastRepeatedsViaReflection( + Message* message, bool expect_extensions_notnull) { + const Reflection* reflection = message->GetReflection(); + + vector<const FieldDescriptor*> output; + reflection->ListFields(*message, &output); + for (int i=0; i<output.size(); ++i) { + const FieldDescriptor* field = output[i]; + if (!field->is_repeated()) continue; + if (field->cpp_type() != FieldDescriptor::CPPTYPE_MESSAGE) continue; + + Message* released = reflection->ReleaseLast(message, field); + if (!field->is_extension() || expect_extensions_notnull) { + ASSERT_TRUE(released != NULL) << "ReleaseLast returned NULL for: " + << field->name(); + } + delete released; + } +} + void TestUtil::ReflectionTester::SwapRepeatedsViaReflection(Message* message) { const Reflection* reflection = message->GetReflection(); @@ -2850,5 +3013,35 @@ void TestUtil::ReflectionTester::SwapRepeatedsViaReflection(Message* message) { } } +void TestUtil::ReflectionTester::ExpectMessagesReleasedViaReflection( + Message* message, + TestUtil::ReflectionTester::MessageReleaseState expected_release_state) { + const Reflection* reflection = message->GetReflection(); + + static const char* fields[] = { + "optionalgroup", + "optional_nested_message", + "optional_foreign_message", + "optional_import_message", + }; + for (int i = 0; i < GOOGLE_ARRAYSIZE(fields); i++) { + const Message& sub_message = reflection->GetMessage(*message, F(fields[i])); + Message* released = reflection->ReleaseMessage(message, F(fields[i])); + switch (expected_release_state) { + case IS_NULL: + EXPECT_TRUE(released == NULL); + break; + case NOT_NULL: + EXPECT_TRUE(released != NULL); + EXPECT_EQ(&sub_message, released); + break; + case CAN_BE_NULL: + break; + } + delete released; + EXPECT_FALSE(reflection->HasField(*message, F(fields[i]))); + } +} + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/test_util.h b/third_party/protobuf/src/google/protobuf/test_util.h index 25165f3..4551957 100644 --- a/third_party/protobuf/src/google/protobuf/test_util.h +++ b/third_party/protobuf/src/google/protobuf/test_util.h @@ -43,13 +43,17 @@ namespace google { namespace protobuf { -namespace unittest = protobuf_unittest; +namespace unittest = ::protobuf_unittest; namespace unittest_import = protobuf_unittest_import; class TestUtil { public: // Set every field in the message to a unique value. static void SetAllFields(unittest::TestAllTypes* message); + static void SetOptionalFields(unittest::TestAllTypes* message); + static void AddRepeatedFields1(unittest::TestAllTypes* message); + static void AddRepeatedFields2(unittest::TestAllTypes* message); + static void SetDefaultFields(unittest::TestAllTypes* message); static void SetAllExtensions(unittest::TestAllExtensions* message); static void SetAllFieldsAndExtensions(unittest::TestFieldOrderings* message); static void SetPackedFields(unittest::TestPackedTypes* message); @@ -104,6 +108,10 @@ class TestUtil { const unittest::TestAllTypes& message); static void ExpectLastRepeatedExtensionsRemoved( const unittest::TestAllExtensions& message); + static void ExpectLastRepeatedsReleased( + const unittest::TestAllTypes& message); + static void ExpectLastRepeatedExtensionsReleased( + const unittest::TestAllExtensions& message); // Check that all repeated fields have had their first and last elements // swapped. @@ -132,8 +140,18 @@ class TestUtil { void ExpectPackedClearViaReflection(const Message& message); void RemoveLastRepeatedsViaReflection(Message* message); + void ReleaseLastRepeatedsViaReflection( + Message* message, bool expect_extensions_notnull); void SwapRepeatedsViaReflection(Message* message); + enum MessageReleaseState { + IS_NULL, + CAN_BE_NULL, + NOT_NULL, + }; + void ExpectMessagesReleasedViaReflection( + Message* message, MessageReleaseState expected_release_state); + private: const FieldDescriptor* F(const string& name); @@ -144,6 +162,7 @@ class TestUtil { const FieldDescriptor* nested_b_; const FieldDescriptor* foreign_c_; const FieldDescriptor* import_d_; + const FieldDescriptor* import_e_; const EnumValueDescriptor* nested_foo_; const EnumValueDescriptor* nested_bar_; diff --git a/third_party/protobuf/src/google/protobuf/test_util_lite.cc b/third_party/protobuf/src/google/protobuf/test_util_lite.cc index d7140e0..9099292 100644 --- a/third_party/protobuf/src/google/protobuf/test_util_lite.cc +++ b/third_party/protobuf/src/google/protobuf/test_util_lite.cc @@ -62,10 +62,12 @@ void TestUtilLite::SetAllFields(unittest::TestAllTypesLite* message) { message->set_optional_string ("115"); message->set_optional_bytes ("116"); - message->mutable_optionalgroup ()->set_a(117); - message->mutable_optional_nested_message ()->set_bb(118); - message->mutable_optional_foreign_message()->set_c(119); - message->mutable_optional_import_message ()->set_d(120); + message->mutable_optionalgroup ()->set_a(117); + message->mutable_optional_nested_message ()->set_bb(118); + message->mutable_optional_foreign_message ()->set_c(119); + message->mutable_optional_import_message ()->set_d(120); + message->mutable_optional_public_import_message()->set_e(126); + message->mutable_optional_lazy_message ()->set_bb(127); message->set_optional_nested_enum (unittest::TestAllTypesLite::BAZ ); message->set_optional_foreign_enum(unittest::FOREIGN_LITE_BAZ ); @@ -94,6 +96,7 @@ void TestUtilLite::SetAllFields(unittest::TestAllTypesLite* message) { message->add_repeated_nested_message ()->set_bb(218); message->add_repeated_foreign_message()->set_c(219); message->add_repeated_import_message ()->set_d(220); + message->add_repeated_lazy_message ()->set_bb(227); message->add_repeated_nested_enum (unittest::TestAllTypesLite::BAR ); message->add_repeated_foreign_enum(unittest::FOREIGN_LITE_BAR ); @@ -121,6 +124,7 @@ void TestUtilLite::SetAllFields(unittest::TestAllTypesLite* message) { message->add_repeated_nested_message ()->set_bb(318); message->add_repeated_foreign_message()->set_c(319); message->add_repeated_import_message ()->set_d(320); + message->add_repeated_lazy_message ()->set_bb(327); message->add_repeated_nested_enum (unittest::TestAllTypesLite::BAZ ); message->add_repeated_foreign_enum(unittest::FOREIGN_LITE_BAZ ); @@ -174,6 +178,7 @@ void TestUtilLite::ModifyRepeatedFields(unittest::TestAllTypesLite* message) { message->mutable_repeated_nested_message (1)->set_bb(518); message->mutable_repeated_foreign_message(1)->set_c(519); message->mutable_repeated_import_message (1)->set_d(520); + message->mutable_repeated_lazy_message (1)->set_bb(527); message->set_repeated_nested_enum (1, unittest::TestAllTypesLite::FOO ); message->set_repeated_foreign_enum(1, unittest::FOREIGN_LITE_FOO ); @@ -201,15 +206,19 @@ void TestUtilLite::ExpectAllFieldsSet( EXPECT_TRUE(message.has_optional_string ()); EXPECT_TRUE(message.has_optional_bytes ()); - EXPECT_TRUE(message.has_optionalgroup ()); - EXPECT_TRUE(message.has_optional_nested_message ()); - EXPECT_TRUE(message.has_optional_foreign_message()); - EXPECT_TRUE(message.has_optional_import_message ()); + EXPECT_TRUE(message.has_optionalgroup ()); + EXPECT_TRUE(message.has_optional_nested_message ()); + EXPECT_TRUE(message.has_optional_foreign_message ()); + EXPECT_TRUE(message.has_optional_import_message ()); + EXPECT_TRUE(message.has_optional_public_import_message()); + EXPECT_TRUE(message.has_optional_lazy_message ()); - EXPECT_TRUE(message.optionalgroup ().has_a()); - EXPECT_TRUE(message.optional_nested_message ().has_bb()); - EXPECT_TRUE(message.optional_foreign_message().has_c()); - EXPECT_TRUE(message.optional_import_message ().has_d()); + EXPECT_TRUE(message.optionalgroup ().has_a()); + EXPECT_TRUE(message.optional_nested_message ().has_bb()); + EXPECT_TRUE(message.optional_foreign_message ().has_c()); + EXPECT_TRUE(message.optional_import_message ().has_d()); + EXPECT_TRUE(message.optional_public_import_message().has_e()); + EXPECT_TRUE(message.optional_lazy_message ().has_bb()); EXPECT_TRUE(message.has_optional_nested_enum ()); EXPECT_TRUE(message.has_optional_foreign_enum()); @@ -232,10 +241,12 @@ void TestUtilLite::ExpectAllFieldsSet( EXPECT_EQ("115", message.optional_string ()); EXPECT_EQ("116", message.optional_bytes ()); - EXPECT_EQ(117, message.optionalgroup ().a()); - EXPECT_EQ(118, message.optional_nested_message ().bb()); - EXPECT_EQ(119, message.optional_foreign_message().c()); - EXPECT_EQ(120, message.optional_import_message ().d()); + EXPECT_EQ(117, message.optionalgroup ().a()); + EXPECT_EQ(118, message.optional_nested_message ().bb()); + EXPECT_EQ(119, message.optional_foreign_message ().c()); + EXPECT_EQ(120, message.optional_import_message ().d()); + EXPECT_EQ(126, message.optional_public_import_message().e()); + EXPECT_EQ(127, message.optional_lazy_message ().bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAZ , message.optional_nested_enum ()); EXPECT_EQ(unittest::FOREIGN_LITE_BAZ , message.optional_foreign_enum()); @@ -264,6 +275,7 @@ void TestUtilLite::ExpectAllFieldsSet( ASSERT_EQ(2, message.repeated_nested_message_size ()); ASSERT_EQ(2, message.repeated_foreign_message_size()); ASSERT_EQ(2, message.repeated_import_message_size ()); + ASSERT_EQ(2, message.repeated_lazy_message_size ()); ASSERT_EQ(2, message.repeated_nested_enum_size ()); ASSERT_EQ(2, message.repeated_foreign_enum_size ()); ASSERT_EQ(2, message.repeated_import_enum_size ()); @@ -289,6 +301,7 @@ void TestUtilLite::ExpectAllFieldsSet( EXPECT_EQ(218, message.repeated_nested_message (0).bb()); EXPECT_EQ(219, message.repeated_foreign_message(0).c()); EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(227, message.repeated_lazy_message (0).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAR , message.repeated_nested_enum (0)); @@ -315,6 +328,7 @@ void TestUtilLite::ExpectAllFieldsSet( EXPECT_EQ(318, message.repeated_nested_message (1).bb()); EXPECT_EQ(319, message.repeated_foreign_message(1).c()); EXPECT_EQ(320, message.repeated_import_message (1).d()); + EXPECT_EQ(327, message.repeated_lazy_message (1).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAZ , message.repeated_nested_enum (1)); EXPECT_EQ(unittest::FOREIGN_LITE_BAZ , message.repeated_foreign_enum(1)); @@ -386,10 +400,12 @@ void TestUtilLite::ExpectClear(const unittest::TestAllTypesLite& message) { EXPECT_FALSE(message.has_optional_string ()); EXPECT_FALSE(message.has_optional_bytes ()); - EXPECT_FALSE(message.has_optionalgroup ()); - EXPECT_FALSE(message.has_optional_nested_message ()); - EXPECT_FALSE(message.has_optional_foreign_message()); - EXPECT_FALSE(message.has_optional_import_message ()); + EXPECT_FALSE(message.has_optionalgroup ()); + EXPECT_FALSE(message.has_optional_nested_message ()); + EXPECT_FALSE(message.has_optional_foreign_message ()); + EXPECT_FALSE(message.has_optional_import_message ()); + EXPECT_FALSE(message.has_optional_public_import_message()); + EXPECT_FALSE(message.has_optional_lazy_message ()); EXPECT_FALSE(message.has_optional_nested_enum ()); EXPECT_FALSE(message.has_optional_foreign_enum()); @@ -414,10 +430,12 @@ void TestUtilLite::ExpectClear(const unittest::TestAllTypesLite& message) { EXPECT_EQ("" , message.optional_bytes ()); // Embedded messages should also be clear. - EXPECT_FALSE(message.optionalgroup ().has_a()); - EXPECT_FALSE(message.optional_nested_message ().has_bb()); - EXPECT_FALSE(message.optional_foreign_message().has_c()); - EXPECT_FALSE(message.optional_import_message ().has_d()); + EXPECT_FALSE(message.optionalgroup ().has_a()); + EXPECT_FALSE(message.optional_nested_message ().has_bb()); + EXPECT_FALSE(message.optional_foreign_message ().has_c()); + EXPECT_FALSE(message.optional_import_message ().has_d()); + EXPECT_FALSE(message.optional_public_import_message().has_e()); + EXPECT_FALSE(message.optional_lazy_message ().has_bb()); EXPECT_EQ(0, message.optionalgroup ().a()); EXPECT_EQ(0, message.optional_nested_message ().bb()); @@ -451,6 +469,7 @@ void TestUtilLite::ExpectClear(const unittest::TestAllTypesLite& message) { EXPECT_EQ(0, message.repeated_nested_message_size ()); EXPECT_EQ(0, message.repeated_foreign_message_size()); EXPECT_EQ(0, message.repeated_import_message_size ()); + EXPECT_EQ(0, message.repeated_lazy_message_size ()); EXPECT_EQ(0, message.repeated_nested_enum_size ()); EXPECT_EQ(0, message.repeated_foreign_enum_size ()); EXPECT_EQ(0, message.repeated_import_enum_size ()); @@ -528,6 +547,7 @@ void TestUtilLite::ExpectRepeatedFieldsModified( ASSERT_EQ(2, message.repeated_nested_message_size ()); ASSERT_EQ(2, message.repeated_foreign_message_size()); ASSERT_EQ(2, message.repeated_import_message_size ()); + ASSERT_EQ(2, message.repeated_lazy_message_size ()); ASSERT_EQ(2, message.repeated_nested_enum_size ()); ASSERT_EQ(2, message.repeated_foreign_enum_size ()); ASSERT_EQ(2, message.repeated_import_enum_size ()); @@ -553,6 +573,7 @@ void TestUtilLite::ExpectRepeatedFieldsModified( EXPECT_EQ(218, message.repeated_nested_message (0).bb()); EXPECT_EQ(219, message.repeated_foreign_message(0).c()); EXPECT_EQ(220, message.repeated_import_message (0).d()); + EXPECT_EQ(227, message.repeated_lazy_message (0).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAR , message.repeated_nested_enum (0)); EXPECT_EQ(unittest::FOREIGN_LITE_BAR , message.repeated_foreign_enum(0)); @@ -580,6 +601,7 @@ void TestUtilLite::ExpectRepeatedFieldsModified( EXPECT_EQ(518, message.repeated_nested_message (1).bb()); EXPECT_EQ(519, message.repeated_foreign_message(1).c()); EXPECT_EQ(520, message.repeated_import_message (1).d()); + EXPECT_EQ(527, message.repeated_lazy_message (1).bb()); EXPECT_EQ(unittest::TestAllTypesLite::FOO , message.repeated_nested_enum (1)); EXPECT_EQ(unittest::FOREIGN_LITE_FOO , message.repeated_foreign_enum(1)); @@ -787,10 +809,12 @@ void TestUtilLite::SetAllExtensions(unittest::TestAllExtensionsLite* message) { message->SetExtension(unittest::optional_string_extension_lite , "115"); message->SetExtension(unittest::optional_bytes_extension_lite , "116"); - message->MutableExtension(unittest::optionalgroup_extension_lite )->set_a(117); - message->MutableExtension(unittest::optional_nested_message_extension_lite )->set_bb(118); - message->MutableExtension(unittest::optional_foreign_message_extension_lite)->set_c(119); - message->MutableExtension(unittest::optional_import_message_extension_lite )->set_d(120); + message->MutableExtension(unittest::optionalgroup_extension_lite )->set_a(117); + message->MutableExtension(unittest::optional_nested_message_extension_lite )->set_bb(118); + message->MutableExtension(unittest::optional_foreign_message_extension_lite )->set_c(119); + message->MutableExtension(unittest::optional_import_message_extension_lite )->set_d(120); + message->MutableExtension(unittest::optional_public_import_message_extension_lite)->set_e(126); + message->MutableExtension(unittest::optional_lazy_message_extension_lite )->set_bb(127); message->SetExtension(unittest::optional_nested_enum_extension_lite , unittest::TestAllTypesLite::BAZ ); message->SetExtension(unittest::optional_foreign_enum_extension_lite, unittest::FOREIGN_LITE_BAZ ); @@ -819,6 +843,7 @@ void TestUtilLite::SetAllExtensions(unittest::TestAllExtensionsLite* message) { message->AddExtension(unittest::repeated_nested_message_extension_lite )->set_bb(218); message->AddExtension(unittest::repeated_foreign_message_extension_lite)->set_c(219); message->AddExtension(unittest::repeated_import_message_extension_lite )->set_d(220); + message->AddExtension(unittest::repeated_lazy_message_extension_lite )->set_bb(227); message->AddExtension(unittest::repeated_nested_enum_extension_lite , unittest::TestAllTypesLite::BAR ); message->AddExtension(unittest::repeated_foreign_enum_extension_lite, unittest::FOREIGN_LITE_BAR ); @@ -846,6 +871,7 @@ void TestUtilLite::SetAllExtensions(unittest::TestAllExtensionsLite* message) { message->AddExtension(unittest::repeated_nested_message_extension_lite )->set_bb(318); message->AddExtension(unittest::repeated_foreign_message_extension_lite)->set_c(319); message->AddExtension(unittest::repeated_import_message_extension_lite )->set_d(320); + message->AddExtension(unittest::repeated_lazy_message_extension_lite )->set_bb(327); message->AddExtension(unittest::repeated_nested_enum_extension_lite , unittest::TestAllTypesLite::BAZ ); message->AddExtension(unittest::repeated_foreign_enum_extension_lite, unittest::FOREIGN_LITE_BAZ ); @@ -900,6 +926,7 @@ void TestUtilLite::ModifyRepeatedExtensions( message->MutableExtension(unittest::repeated_nested_message_extension_lite , 1)->set_bb(518); message->MutableExtension(unittest::repeated_foreign_message_extension_lite, 1)->set_c(519); message->MutableExtension(unittest::repeated_import_message_extension_lite , 1)->set_d(520); + message->MutableExtension(unittest::repeated_lazy_message_extension_lite , 1)->set_bb(527); message->SetExtension(unittest::repeated_nested_enum_extension_lite , 1, unittest::TestAllTypesLite::FOO ); message->SetExtension(unittest::repeated_foreign_enum_extension_lite, 1, unittest::FOREIGN_LITE_FOO ); @@ -927,15 +954,19 @@ void TestUtilLite::ExpectAllExtensionsSet( EXPECT_TRUE(message.HasExtension(unittest::optional_string_extension_lite )); EXPECT_TRUE(message.HasExtension(unittest::optional_bytes_extension_lite )); - EXPECT_TRUE(message.HasExtension(unittest::optionalgroup_extension_lite )); - EXPECT_TRUE(message.HasExtension(unittest::optional_nested_message_extension_lite )); - EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_message_extension_lite)); - EXPECT_TRUE(message.HasExtension(unittest::optional_import_message_extension_lite )); + EXPECT_TRUE(message.HasExtension(unittest::optionalgroup_extension_lite )); + EXPECT_TRUE(message.HasExtension(unittest::optional_nested_message_extension_lite )); + EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_message_extension_lite )); + EXPECT_TRUE(message.HasExtension(unittest::optional_import_message_extension_lite )); + EXPECT_TRUE(message.HasExtension(unittest::optional_public_import_message_extension_lite)); + EXPECT_TRUE(message.HasExtension(unittest::optional_lazy_message_extension_lite )); - EXPECT_TRUE(message.GetExtension(unittest::optionalgroup_extension_lite ).has_a()); - EXPECT_TRUE(message.GetExtension(unittest::optional_nested_message_extension_lite ).has_bb()); - EXPECT_TRUE(message.GetExtension(unittest::optional_foreign_message_extension_lite).has_c()); - EXPECT_TRUE(message.GetExtension(unittest::optional_import_message_extension_lite ).has_d()); + EXPECT_TRUE(message.GetExtension(unittest::optionalgroup_extension_lite ).has_a()); + EXPECT_TRUE(message.GetExtension(unittest::optional_nested_message_extension_lite ).has_bb()); + EXPECT_TRUE(message.GetExtension(unittest::optional_foreign_message_extension_lite ).has_c()); + EXPECT_TRUE(message.GetExtension(unittest::optional_import_message_extension_lite ).has_d()); + EXPECT_TRUE(message.GetExtension(unittest::optional_public_import_message_extension_lite).has_e()); + EXPECT_TRUE(message.GetExtension(unittest::optional_lazy_message_extension_lite ).has_bb()); EXPECT_TRUE(message.HasExtension(unittest::optional_nested_enum_extension_lite )); EXPECT_TRUE(message.HasExtension(unittest::optional_foreign_enum_extension_lite)); @@ -958,10 +989,12 @@ void TestUtilLite::ExpectAllExtensionsSet( EXPECT_EQ("115", message.GetExtension(unittest::optional_string_extension_lite )); EXPECT_EQ("116", message.GetExtension(unittest::optional_bytes_extension_lite )); - EXPECT_EQ(117, message.GetExtension(unittest::optionalgroup_extension_lite ).a()); - EXPECT_EQ(118, message.GetExtension(unittest::optional_nested_message_extension_lite ).bb()); - EXPECT_EQ(119, message.GetExtension(unittest::optional_foreign_message_extension_lite).c()); - EXPECT_EQ(120, message.GetExtension(unittest::optional_import_message_extension_lite ).d()); + EXPECT_EQ(117, message.GetExtension(unittest::optionalgroup_extension_lite ).a()); + EXPECT_EQ(118, message.GetExtension(unittest::optional_nested_message_extension_lite ).bb()); + EXPECT_EQ(119, message.GetExtension(unittest::optional_foreign_message_extension_lite ).c()); + EXPECT_EQ(120, message.GetExtension(unittest::optional_import_message_extension_lite ).d()); + EXPECT_EQ(126, message.GetExtension(unittest::optional_public_import_message_extension_lite).e()); + EXPECT_EQ(127, message.GetExtension(unittest::optional_lazy_message_extension_lite ).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAZ , message.GetExtension(unittest::optional_nested_enum_extension_lite )); EXPECT_EQ(unittest::FOREIGN_LITE_BAZ , message.GetExtension(unittest::optional_foreign_enum_extension_lite)); @@ -990,6 +1023,7 @@ void TestUtilLite::ExpectAllExtensionsSet( ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_message_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_message_extension_lite)); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_message_extension_lite )); + ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_lazy_message_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_enum_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_enum_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_enum_extension_lite )); @@ -1015,6 +1049,7 @@ void TestUtilLite::ExpectAllExtensionsSet( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension_lite , 0).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension_lite, 0).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension_lite , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension_lite , 0).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAR , message.GetExtension(unittest::repeated_nested_enum_extension_lite , 0)); EXPECT_EQ(unittest::FOREIGN_LITE_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension_lite, 0)); @@ -1041,6 +1076,7 @@ void TestUtilLite::ExpectAllExtensionsSet( EXPECT_EQ(318, message.GetExtension(unittest::repeated_nested_message_extension_lite , 1).bb()); EXPECT_EQ(319, message.GetExtension(unittest::repeated_foreign_message_extension_lite, 1).c()); EXPECT_EQ(320, message.GetExtension(unittest::repeated_import_message_extension_lite , 1).d()); + EXPECT_EQ(327, message.GetExtension(unittest::repeated_lazy_message_extension_lite , 1).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAZ , message.GetExtension(unittest::repeated_nested_enum_extension_lite , 1)); EXPECT_EQ(unittest::FOREIGN_LITE_BAZ , message.GetExtension(unittest::repeated_foreign_enum_extension_lite, 1)); @@ -1118,10 +1154,12 @@ void TestUtilLite::ExpectExtensionsClear( EXPECT_FALSE(message.HasExtension(unittest::optional_string_extension_lite )); EXPECT_FALSE(message.HasExtension(unittest::optional_bytes_extension_lite )); - EXPECT_FALSE(message.HasExtension(unittest::optionalgroup_extension_lite )); - EXPECT_FALSE(message.HasExtension(unittest::optional_nested_message_extension_lite )); - EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_message_extension_lite)); - EXPECT_FALSE(message.HasExtension(unittest::optional_import_message_extension_lite )); + EXPECT_FALSE(message.HasExtension(unittest::optionalgroup_extension_lite )); + EXPECT_FALSE(message.HasExtension(unittest::optional_nested_message_extension_lite )); + EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_message_extension_lite )); + EXPECT_FALSE(message.HasExtension(unittest::optional_import_message_extension_lite )); + EXPECT_FALSE(message.HasExtension(unittest::optional_public_import_message_extension_lite)); + EXPECT_FALSE(message.HasExtension(unittest::optional_lazy_message_extension_lite )); EXPECT_FALSE(message.HasExtension(unittest::optional_nested_enum_extension_lite )); EXPECT_FALSE(message.HasExtension(unittest::optional_foreign_enum_extension_lite)); @@ -1146,15 +1184,19 @@ void TestUtilLite::ExpectExtensionsClear( EXPECT_EQ("" , message.GetExtension(unittest::optional_bytes_extension_lite )); // Embedded messages should also be clear. - EXPECT_FALSE(message.GetExtension(unittest::optionalgroup_extension_lite ).has_a()); - EXPECT_FALSE(message.GetExtension(unittest::optional_nested_message_extension_lite ).has_bb()); - EXPECT_FALSE(message.GetExtension(unittest::optional_foreign_message_extension_lite).has_c()); - EXPECT_FALSE(message.GetExtension(unittest::optional_import_message_extension_lite ).has_d()); - - EXPECT_EQ(0, message.GetExtension(unittest::optionalgroup_extension_lite ).a()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_nested_message_extension_lite ).bb()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_foreign_message_extension_lite).c()); - EXPECT_EQ(0, message.GetExtension(unittest::optional_import_message_extension_lite ).d()); + EXPECT_FALSE(message.GetExtension(unittest::optionalgroup_extension_lite ).has_a()); + EXPECT_FALSE(message.GetExtension(unittest::optional_nested_message_extension_lite ).has_bb()); + EXPECT_FALSE(message.GetExtension(unittest::optional_foreign_message_extension_lite ).has_c()); + EXPECT_FALSE(message.GetExtension(unittest::optional_import_message_extension_lite ).has_d()); + EXPECT_FALSE(message.GetExtension(unittest::optional_public_import_message_extension_lite).has_e()); + EXPECT_FALSE(message.GetExtension(unittest::optional_lazy_message_extension_lite ).has_bb()); + + EXPECT_EQ(0, message.GetExtension(unittest::optionalgroup_extension_lite ).a()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_nested_message_extension_lite ).bb()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_foreign_message_extension_lite ).c()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_import_message_extension_lite ).d()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_public_import_message_extension_lite).e()); + EXPECT_EQ(0, message.GetExtension(unittest::optional_lazy_message_extension_lite ).bb()); // Enums without defaults are set to the first value in the enum. EXPECT_EQ(unittest::TestAllTypesLite::FOO , message.GetExtension(unittest::optional_nested_enum_extension_lite )); @@ -1183,6 +1225,7 @@ void TestUtilLite::ExpectExtensionsClear( EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_nested_message_extension_lite )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_foreign_message_extension_lite)); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_import_message_extension_lite )); + EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_lazy_message_extension_lite )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_nested_enum_extension_lite )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_foreign_enum_extension_lite )); EXPECT_EQ(0, message.ExtensionSize(unittest::repeated_import_enum_extension_lite )); @@ -1260,6 +1303,7 @@ void TestUtilLite::ExpectRepeatedExtensionsModified( ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_message_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_message_extension_lite)); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_message_extension_lite )); + ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_lazy_message_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_nested_enum_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_foreign_enum_extension_lite )); ASSERT_EQ(2, message.ExtensionSize(unittest::repeated_import_enum_extension_lite )); @@ -1285,6 +1329,7 @@ void TestUtilLite::ExpectRepeatedExtensionsModified( EXPECT_EQ(218, message.GetExtension(unittest::repeated_nested_message_extension_lite , 0).bb()); EXPECT_EQ(219, message.GetExtension(unittest::repeated_foreign_message_extension_lite, 0).c()); EXPECT_EQ(220, message.GetExtension(unittest::repeated_import_message_extension_lite , 0).d()); + EXPECT_EQ(227, message.GetExtension(unittest::repeated_lazy_message_extension_lite , 0).bb()); EXPECT_EQ(unittest::TestAllTypesLite::BAR , message.GetExtension(unittest::repeated_nested_enum_extension_lite , 0)); EXPECT_EQ(unittest::FOREIGN_LITE_BAR , message.GetExtension(unittest::repeated_foreign_enum_extension_lite, 0)); @@ -1312,6 +1357,7 @@ void TestUtilLite::ExpectRepeatedExtensionsModified( EXPECT_EQ(518, message.GetExtension(unittest::repeated_nested_message_extension_lite , 1).bb()); EXPECT_EQ(519, message.GetExtension(unittest::repeated_foreign_message_extension_lite, 1).c()); EXPECT_EQ(520, message.GetExtension(unittest::repeated_import_message_extension_lite , 1).d()); + EXPECT_EQ(527, message.GetExtension(unittest::repeated_lazy_message_extension_lite , 1).bb()); EXPECT_EQ(unittest::TestAllTypesLite::FOO , message.GetExtension(unittest::repeated_nested_enum_extension_lite , 1)); EXPECT_EQ(unittest::FOREIGN_LITE_FOO , message.GetExtension(unittest::repeated_foreign_enum_extension_lite, 1)); diff --git a/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_data.txt b/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_data.txt index feea8f7..bbe5882 100644 --- a/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_data.txt +++ b/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_data.txt @@ -30,6 +30,12 @@ optional_foreign_enum: FOREIGN_BAZ optional_import_enum: IMPORT_BAZ optional_string_piece: "124" optional_cord: "125" +optional_public_import_message { + e: 126 +} +optional_lazy_message { + bb: 127 +} repeated_int32: 201 repeated_int32: 301 repeated_int64: 202 @@ -94,6 +100,12 @@ repeated_string_piece: "224" repeated_string_piece: "324" repeated_cord: "225" repeated_cord: "325" +repeated_lazy_message { + bb: 227 +} +repeated_lazy_message { + bb: 327 +} default_int32: 401 default_int64: 402 default_uint32: 403 diff --git a/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_extensions_data.txt b/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_extensions_data.txt index 057beae..0a217f0 100644 --- a/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_extensions_data.txt +++ b/third_party/protobuf/src/google/protobuf/testdata/text_format_unittest_extensions_data.txt @@ -30,6 +30,12 @@ [protobuf_unittest.optional_import_enum_extension]: IMPORT_BAZ [protobuf_unittest.optional_string_piece_extension]: "124" [protobuf_unittest.optional_cord_extension]: "125" +[protobuf_unittest.optional_public_import_message_extension] { + e: 126 +} +[protobuf_unittest.optional_lazy_message_extension] { + bb: 127 +} [protobuf_unittest.repeated_int32_extension]: 201 [protobuf_unittest.repeated_int32_extension]: 301 [protobuf_unittest.repeated_int64_extension]: 202 @@ -94,6 +100,12 @@ [protobuf_unittest.repeated_string_piece_extension]: "324" [protobuf_unittest.repeated_cord_extension]: "225" [protobuf_unittest.repeated_cord_extension]: "325" +[protobuf_unittest.repeated_lazy_message_extension] { + bb: 227 +} +[protobuf_unittest.repeated_lazy_message_extension] { + bb: 327 +} [protobuf_unittest.default_int32_extension]: 401 [protobuf_unittest.default_int64_extension]: 402 [protobuf_unittest.default_uint32_extension]: 403 diff --git a/third_party/protobuf/src/google/protobuf/testing/googletest.cc b/third_party/protobuf/src/google/protobuf/testing/googletest.cc index cd094d0..a8da6b1 100644 --- a/third_party/protobuf/src/google/protobuf/testing/googletest.cc +++ b/third_party/protobuf/src/google/protobuf/testing/googletest.cc @@ -223,16 +223,17 @@ ScopedMemoryLog::~ScopedMemoryLog() { active_log_ = NULL; } -const vector<string>& ScopedMemoryLog::GetMessages(LogLevel dummy) const { - GOOGLE_CHECK_EQ(dummy, ERROR); - return messages_; +const vector<string>& ScopedMemoryLog::GetMessages(LogLevel level) { + GOOGLE_CHECK(level == ERROR || + level == WARNING); + return messages_[level]; } void ScopedMemoryLog::HandleLog(LogLevel level, const char* filename, int line, const string& message) { GOOGLE_CHECK(active_log_ != NULL); - if (level == ERROR) { - active_log_->messages_.push_back(message); + if (level == ERROR || level == WARNING) { + active_log_->messages_[level].push_back(message); } } diff --git a/third_party/protobuf/src/google/protobuf/testing/googletest.h b/third_party/protobuf/src/google/protobuf/testing/googletest.h index 71444c9..6b17ae4 100644 --- a/third_party/protobuf/src/google/protobuf/testing/googletest.h +++ b/third_party/protobuf/src/google/protobuf/testing/googletest.h @@ -34,6 +34,7 @@ #ifndef GOOGLE_PROTOBUF_GOOGLETEST_H__ #define GOOGLE_PROTOBUF_GOOGLETEST_H__ +#include <map> #include <vector> #include <google/protobuf/stubs/common.h> @@ -60,6 +61,7 @@ string GetCapturedTestStderr(); // ScopedMemoryLog refers to LOGLEVEL_ERROR as just ERROR. #undef ERROR // defend against promiscuous windows.h static const LogLevel ERROR = LOGLEVEL_ERROR; +static const LogLevel WARNING = LOGLEVEL_WARNING; // Receives copies of all LOG(ERROR) messages while in scope. Sample usage: // { @@ -74,14 +76,11 @@ class ScopedMemoryLog { ScopedMemoryLog(); virtual ~ScopedMemoryLog(); - // Fetches all messages logged. The internal version of this class - // would only fetch messages at the given security level, but the protobuf - // open source version ignores the argument since we always pass ERROR - // anyway. - const vector<string>& GetMessages(LogLevel dummy) const; + // Fetches all messages with the given severity level. + const vector<string>& GetMessages(LogLevel error); private: - vector<string> messages_; + map<LogLevel, vector<string> > messages_; LogHandler* old_handler_; static void HandleLog(LogLevel level, const char* filename, int line, diff --git a/third_party/protobuf/src/google/protobuf/text_format.cc b/third_party/protobuf/src/google/protobuf/text_format.cc index 1494ebd..1ee3df5 100644 --- a/third_party/protobuf/src/google/protobuf/text_format.cc +++ b/third_party/protobuf/src/google/protobuf/text_format.cc @@ -49,6 +49,8 @@ #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/io/tokenizer.h> #include <google/protobuf/stubs/strutil.h> +#include <google/protobuf/stubs/map-util.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -94,6 +96,73 @@ void Message::PrintDebugString() const { // =========================================================================== +// Implementation of the parse information tree class. +TextFormat::ParseInfoTree::ParseInfoTree() { } + +TextFormat::ParseInfoTree::~ParseInfoTree() { + // Remove any nested information trees, as they are owned by this tree. + for (NestedMap::iterator it = nested_.begin(); it != nested_.end(); ++it) { + STLDeleteElements(&(it->second)); + } +} + +void TextFormat::ParseInfoTree::RecordLocation( + const FieldDescriptor* field, + TextFormat::ParseLocation location) { + locations_[field].push_back(location); +} + +TextFormat::ParseInfoTree* TextFormat::ParseInfoTree::CreateNested( + const FieldDescriptor* field) { + // Owned by us in the map. + TextFormat::ParseInfoTree* instance = new TextFormat::ParseInfoTree(); + vector<TextFormat::ParseInfoTree*>* trees = &nested_[field]; + GOOGLE_CHECK(trees); + trees->push_back(instance); + return instance; +} + +void CheckFieldIndex(const FieldDescriptor* field, int index) { + if (field == NULL) { return; } + + if (field->is_repeated() && index == -1) { + GOOGLE_LOG(DFATAL) << "Index must be in range of repeated field values. " + << "Field: " << field->name(); + } else if (!field->is_repeated() && index != -1) { + GOOGLE_LOG(DFATAL) << "Index must be -1 for singular fields." + << "Field: " << field->name(); + } +} + +TextFormat::ParseLocation TextFormat::ParseInfoTree::GetLocation( + const FieldDescriptor* field, int index) const { + CheckFieldIndex(field, index); + if (index == -1) { index = 0; } + + const vector<TextFormat::ParseLocation>* locations = + FindOrNull(locations_, field); + if (locations == NULL || index >= locations->size()) { + return TextFormat::ParseLocation(); + } + + return (*locations)[index]; +} + +TextFormat::ParseInfoTree* TextFormat::ParseInfoTree::GetTreeForNested( + const FieldDescriptor* field, int index) const { + CheckFieldIndex(field, index); + if (index == -1) { index = 0; } + + const vector<TextFormat::ParseInfoTree*>* trees = FindOrNull(nested_, field); + if (trees == NULL || index >= trees->size()) { + return NULL; + } + + return (*trees)[index]; +} + + +// =========================================================================== // Internal class for parsing an ASCII representation of a Protocol Message. // This class makes use of the Protocol Message compiler's tokenizer found // in //google/protobuf/io/tokenizer.h. Note that class's Parse @@ -120,13 +189,17 @@ class TextFormat::Parser::ParserImpl { io::ZeroCopyInputStream* input_stream, io::ErrorCollector* error_collector, TextFormat::Finder* finder, - SingularOverwritePolicy singular_overwrite_policy) + ParseInfoTree* parse_info_tree, + SingularOverwritePolicy singular_overwrite_policy, + bool allow_unknown_field) : error_collector_(error_collector), finder_(finder), + parse_info_tree_(parse_info_tree), tokenizer_error_collector_(this), tokenizer_(input_stream, &tokenizer_error_collector_), root_message_type_(root_message_type), singular_overwrite_policy_(singular_overwrite_policy), + allow_unknown_field_(allow_unknown_field), had_errors_(false) { // For backwards-compatibility with proto1, we need to allow the 'f' suffix // for floats. @@ -240,6 +313,8 @@ class TextFormat::Parser::ParserImpl { string field_name; const FieldDescriptor* field = NULL; + int start_line = tokenizer_.current().line; + int start_column = tokenizer_.current().column; if (TryConsume("[")) { // Extension. @@ -257,10 +332,16 @@ class TextFormat::Parser::ParserImpl { : reflection->FindKnownExtensionByName(field_name)); if (field == NULL) { - ReportError("Extension \"" + field_name + "\" is not defined or " - "is not an extension of \"" + - descriptor->full_name() + "\"."); - return false; + if (!allow_unknown_field_) { + ReportError("Extension \"" + field_name + "\" is not defined or " + "is not an extension of \"" + + descriptor->full_name() + "\"."); + return false; + } else { + ReportWarning("Extension \"" + field_name + "\" is not defined or " + "is not an extension of \"" + + descriptor->full_name() + "\"."); + } } } else { DO(ConsumeIdentifier(&field_name)); @@ -285,9 +366,30 @@ class TextFormat::Parser::ParserImpl { } if (field == NULL) { - ReportError("Message type \"" + descriptor->full_name() + - "\" has no field named \"" + field_name + "\"."); - return false; + if (!allow_unknown_field_) { + ReportError("Message type \"" + descriptor->full_name() + + "\" has no field named \"" + field_name + "\"."); + return false; + } else { + ReportWarning("Message type \"" + descriptor->full_name() + + "\" has no field named \"" + field_name + "\"."); + } + } + } + + // Skips unknown field. + if (field == NULL) { + GOOGLE_CHECK(allow_unknown_field_); + // Try to guess the type of this field. + // If this field is not a message, there should be a ":" between the + // field name and the field value and also the field value should not + // start with "{" or "<" which indicates the begining of a message body. + // If there is no ":" or there is a "{" or "<" after ":", this field has + // to be a message or the input is ill-formed. + if (TryConsume(":") && !LookingAt("{") && !LookingAt("<")) { + return SkipFieldValue(); + } else { + return SkipFieldMessage(); } } @@ -329,12 +431,61 @@ class TextFormat::Parser::ParserImpl { + field_name + "\""); } + // If a parse info tree exists, add the location for the parsed + // field. + if (parse_info_tree_ != NULL) { + parse_info_tree_->RecordLocation(field, + ParseLocation(start_line, start_column)); + } + + return true; + } + + // Skips the next field including the field's name and value. + bool SkipField() { + string field_name; + if (TryConsume("[")) { + // Extension name. + DO(ConsumeIdentifier(&field_name)); + while (TryConsume(".")) { + string part; + DO(ConsumeIdentifier(&part)); + field_name += "."; + field_name += part; + } + DO(Consume("]")); + } else { + DO(ConsumeIdentifier(&field_name)); + } + + // Try to guess the type of this field. + // If this field is not a message, there should be a ":" between the + // field name and the field value and also the field value should not + // start with "{" or "<" which indicates the begining of a message body. + // If there is no ":" or there is a "{" or "<" after ":", this field has + // to be a message or the input is ill-formed. + if (TryConsume(":") && !LookingAt("{") && !LookingAt("<")) { + DO(SkipFieldValue()); + } else { + DO(SkipFieldMessage()); + } + // For historical reasons, fields may optionally be separated by commas or + // semicolons. + TryConsume(";") || TryConsume(","); return true; } bool ConsumeFieldMessage(Message* message, const Reflection* reflection, const FieldDescriptor* field) { + + // If the parse information tree is not NULL, create a nested one + // for the nested message. + ParseInfoTree* parent = parse_info_tree_; + if (parent != NULL) { + parse_info_tree_ = parent->CreateNested(field); + } + string delimeter; if (TryConsume("<")) { delimeter = ">"; @@ -349,6 +500,26 @@ class TextFormat::Parser::ParserImpl { DO(ConsumeMessage(reflection->MutableMessage(message, field), delimeter)); } + + // Reset the parse information tree. + parse_info_tree_ = parent; + return true; + } + + // Skips the whole body of a message including the begining delimeter and + // the ending delimeter. + bool SkipFieldMessage() { + string delimeter; + if (TryConsume("<")) { + delimeter = ">"; + } else { + DO(Consume("{")); + delimeter = "}"; + } + while (!LookingAt(">") && !LookingAt("}")) { + DO(SkipField()); + } + DO(Consume(delimeter)); return true; } @@ -479,6 +650,60 @@ class TextFormat::Parser::ParserImpl { return true; } + bool SkipFieldValue() { + if (LookingAtType(io::Tokenizer::TYPE_STRING)) { + while (LookingAtType(io::Tokenizer::TYPE_STRING)) { + tokenizer_.Next(); + } + return true; + } + // Possible field values other than string: + // 12345 => TYPE_INTEGER + // -12345 => TYPE_SYMBOL + TYPE_INTEGER + // 1.2345 => TYPE_FLOAT + // -1.2345 => TYPE_SYMBOL + TYPE_FLOAT + // inf => TYPE_IDENTIFIER + // -inf => TYPE_SYMBOL + TYPE_IDENTIFIER + // TYPE_INTEGER => TYPE_IDENTIFIER + // Divides them into two group, one with TYPE_SYMBOL + // and the other without: + // Group one: + // 12345 => TYPE_INTEGER + // 1.2345 => TYPE_FLOAT + // inf => TYPE_IDENTIFIER + // TYPE_INTEGER => TYPE_IDENTIFIER + // Group two: + // -12345 => TYPE_SYMBOL + TYPE_INTEGER + // -1.2345 => TYPE_SYMBOL + TYPE_FLOAT + // -inf => TYPE_SYMBOL + TYPE_IDENTIFIER + // As we can see, the field value consists of an optional '-' and one of + // TYPE_INTEGER, TYPE_FLOAT and TYPE_IDENTIFIER. + bool has_minus = TryConsume("-"); + if (!LookingAtType(io::Tokenizer::TYPE_INTEGER) && + !LookingAtType(io::Tokenizer::TYPE_FLOAT) && + !LookingAtType(io::Tokenizer::TYPE_IDENTIFIER)) { + return false; + } + // Combination of '-' and TYPE_IDENTIFIER may result in an invalid field + // value while other combinations all generate valid values. + // We check if the value of this combination is valid here. + // TYPE_IDENTIFIER after a '-' should be one of the float values listed + // below: + // inf, inff, infinity, nan + if (has_minus && LookingAtType(io::Tokenizer::TYPE_IDENTIFIER)) { + string text = tokenizer_.current().text; + LowerString(&text); + if (text != "inf" && + text != "infinity" && + text != "nan") { + ReportError("Invalid float number: " + text); + return false; + } + } + tokenizer_.Next(); + return true; + } + // Returns true if the current token's text is equal to that specified. bool LookingAt(const string& text) { return tokenizer_.current().text == text; @@ -596,7 +821,8 @@ class TextFormat::Parser::ParserImpl { } else if (LookingAtType(io::Tokenizer::TYPE_IDENTIFIER)) { string text = tokenizer_.current().text; LowerString(&text); - if (text == "inf" || text == "infinity") { + if (text == "inf" || + text == "infinity") { *value = std::numeric_limits<double>::infinity(); tokenizer_.Next(); } else if (text == "nan") { @@ -670,10 +896,12 @@ class TextFormat::Parser::ParserImpl { io::ErrorCollector* error_collector_; TextFormat::Finder* finder_; + ParseInfoTree* parse_info_tree_; ParserErrorCollector tokenizer_error_collector_; io::Tokenizer tokenizer_; const Descriptor* root_message_type_; SingularOverwritePolicy singular_overwrite_policy_; + bool allow_unknown_field_; bool had_errors_; }; @@ -699,7 +927,7 @@ class TextFormat::Printer::TextGenerator { ~TextGenerator() { // Only BackUp() if we're sure we've successfully called Next() at least // once. - if (buffer_size_ > 0) { + if (!failed_ && buffer_size_ > 0) { output_->BackUp(buffer_size_); } } @@ -809,7 +1037,9 @@ TextFormat::Finder::~Finder() { TextFormat::Parser::Parser() : error_collector_(NULL), finder_(NULL), - allow_partial_(false) { + parse_info_tree_(NULL), + allow_partial_(false), + allow_unknown_field_(false) { } TextFormat::Parser::~Parser() {} @@ -818,7 +1048,9 @@ bool TextFormat::Parser::Parse(io::ZeroCopyInputStream* input, Message* output) { output->Clear(); ParserImpl parser(output->GetDescriptor(), input, error_collector_, - finder_, ParserImpl::FORBID_SINGULAR_OVERWRITES); + finder_, parse_info_tree_, + ParserImpl::FORBID_SINGULAR_OVERWRITES, + allow_unknown_field_); return MergeUsingImpl(input, output, &parser); } @@ -831,7 +1063,9 @@ bool TextFormat::Parser::ParseFromString(const string& input, bool TextFormat::Parser::Merge(io::ZeroCopyInputStream* input, Message* output) { ParserImpl parser(output->GetDescriptor(), input, error_collector_, - finder_, ParserImpl::ALLOW_SINGULAR_OVERWRITES); + finder_, parse_info_tree_, + ParserImpl::ALLOW_SINGULAR_OVERWRITES, + allow_unknown_field_); return MergeUsingImpl(input, output, &parser); } @@ -861,7 +1095,9 @@ bool TextFormat::Parser::ParseFieldValueFromString( Message* output) { io::ArrayInputStream input_stream(input.data(), input.size()); ParserImpl parser(output->GetDescriptor(), &input_stream, error_collector_, - finder_, ParserImpl::ALLOW_SINGULAR_OVERWRITES); + finder_, parse_info_tree_, + ParserImpl::ALLOW_SINGULAR_OVERWRITES, + allow_unknown_field_); return parser.ParseField(field, output); } diff --git a/third_party/protobuf/src/google/protobuf/text_format.h b/third_party/protobuf/src/google/protobuf/text_format.h index 03c0491..0974139 100644 --- a/third_party/protobuf/src/google/protobuf/text_format.h +++ b/third_party/protobuf/src/google/protobuf/text_format.h @@ -38,7 +38,10 @@ #ifndef GOOGLE_PROTOBUF_TEXT_FORMAT_H__ #define GOOGLE_PROTOBUF_TEXT_FORMAT_H__ +#include <map> #include <string> +#include <vector> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/message.h> #include <google/protobuf/descriptor.h> @@ -220,6 +223,57 @@ class LIBPROTOBUF_EXPORT TextFormat { const string& name) const = 0; }; + // A location in the parsed text. + struct ParseLocation { + int line; + int column; + + ParseLocation() : line(-1), column(-1) {} + ParseLocation(int line_param, int column_param) + : line(line_param), column(column_param) {} + }; + + // Data structure which is populated with the locations of each field + // value parsed from the text. + class LIBPROTOBUF_EXPORT ParseInfoTree { + public: + ParseInfoTree(); + ~ParseInfoTree(); + + // Returns the parse location for index-th value of the field in the parsed + // text. If none exists, returns a location with line = -1. Index should be + // -1 for not-repeated fields. + ParseLocation GetLocation(const FieldDescriptor* field, int index) const; + + // Returns the parse info tree for the given field, which must be a message + // type. The nested information tree is owned by the root tree and will be + // deleted when it is deleted. + ParseInfoTree* GetTreeForNested(const FieldDescriptor* field, + int index) const; + + private: + // Allow the text format parser to record information into the tree. + friend class TextFormat; + + // Records the starting location of a single value for a field. + void RecordLocation(const FieldDescriptor* field, ParseLocation location); + + // Create and records a nested tree for a nested message field. + ParseInfoTree* CreateNested(const FieldDescriptor* field); + + // Defines the map from the index-th field descriptor to its parse location. + typedef map<const FieldDescriptor*, vector<ParseLocation> > LocationMap; + + // Defines the map from the index-th field descriptor to the nested parse + // info tree. + typedef map<const FieldDescriptor*, vector<ParseInfoTree*> > NestedMap; + + LocationMap locations_; + NestedMap nested_; + + GOOGLE_DISALLOW_EVIL_CONSTRUCTORS(ParseInfoTree); + }; + // For more control over parsing, use this class. class LIBPROTOBUF_EXPORT Parser { public: @@ -248,6 +302,12 @@ class LIBPROTOBUF_EXPORT TextFormat { finder_ = finder; } + // Sets where location information about the parse will be written. If NULL + // (the default), then no location will be written. + void WriteLocationsTo(ParseInfoTree* tree) { + parse_info_tree_ = tree; + } + // Normally parsing fails if, after parsing, output->IsInitialized() // returns false. Call AllowPartialMessage(true) to skip this check. void AllowPartialMessage(bool allow) { @@ -259,6 +319,7 @@ class LIBPROTOBUF_EXPORT TextFormat { const FieldDescriptor* field, Message* output); + private: // Forward declaration of an internal class used to parse text // representations (see text_format.cc for implementation). @@ -272,7 +333,9 @@ class LIBPROTOBUF_EXPORT TextFormat { io::ErrorCollector* error_collector_; Finder* finder_; + ParseInfoTree* parse_info_tree_; bool allow_partial_; + bool allow_unknown_field_; }; private: diff --git a/third_party/protobuf/src/google/protobuf/text_format_unittest.cc b/third_party/protobuf/src/google/protobuf/text_format_unittest.cc index 00ea8c3..353a5e4 100644 --- a/third_party/protobuf/src/google/protobuf/text_format_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/text_format_unittest.cc @@ -339,6 +339,21 @@ TEST_F(TextFormatTest, PrintMessageSingleLine) { text); } +TEST_F(TextFormatTest, PrintBufferTooSmall) { + // Test printing a message to a buffer that is too small. + + protobuf_unittest::TestAllTypes message; + + message.add_repeated_string("abc"); + message.add_repeated_string("def"); + + char buffer[1] = ""; + io::ArrayOutputStream output_stream(buffer, 1); + EXPECT_FALSE(TextFormat::Print(message, &output_stream)); + EXPECT_EQ(buffer[0], 'r'); + EXPECT_EQ(output_stream.ByteCount(), 1); +} + TEST_F(TextFormatTest, ParseBasic) { io::ArrayInputStream input_stream(proto_debug_string_.data(), proto_debug_string_.size()); @@ -728,6 +743,25 @@ class TextFormatParserTest : public testing::Test { error_collector.text_); } + void ExpectSuccessAndTree(const string& input, Message* proto, + TextFormat::ParseInfoTree* info_tree) { + TextFormat::Parser parser; + MockErrorCollector error_collector; + parser.RecordErrorsTo(&error_collector); + parser.WriteLocationsTo(info_tree); + + EXPECT_TRUE(parser.ParseFromString(input, proto)); + } + + void ExpectLocation(TextFormat::ParseInfoTree* tree, + const Descriptor* d, const string& field_name, + int index, int line, int column) { + TextFormat::ParseLocation location = tree->GetLocation( + d->FindFieldByName(field_name), index); + EXPECT_EQ(line, location.line); + EXPECT_EQ(column, location.column); + } + // An error collector which simply concatenates all its errors into a big // block of text which can be checked. class MockErrorCollector : public io::ErrorCollector { @@ -749,6 +783,71 @@ class TextFormatParserTest : public testing::Test { }; }; +TEST_F(TextFormatParserTest, ParseInfoTreeBuilding) { + scoped_ptr<unittest::TestAllTypes> message(new unittest::TestAllTypes); + const Descriptor* d = message->GetDescriptor(); + + string stringData = + "optional_int32: 1\n" + "optional_int64: 2\n" + " optional_double: 2.4\n" + "repeated_int32: 5\n" + "repeated_int32: 10\n" + "optional_nested_message <\n" + " bb: 78\n" + ">\n" + "repeated_nested_message <\n" + " bb: 79\n" + ">\n" + "repeated_nested_message <\n" + " bb: 80\n" + ">"; + + + TextFormat::ParseInfoTree tree; + ExpectSuccessAndTree(stringData, message.get(), &tree); + + // Verify that the tree has the correct positions. + ExpectLocation(&tree, d, "optional_int32", -1, 0, 0); + ExpectLocation(&tree, d, "optional_int64", -1, 1, 0); + ExpectLocation(&tree, d, "optional_double", -1, 2, 2); + + ExpectLocation(&tree, d, "repeated_int32", 0, 3, 0); + ExpectLocation(&tree, d, "repeated_int32", 1, 4, 0); + + ExpectLocation(&tree, d, "optional_nested_message", -1, 5, 0); + ExpectLocation(&tree, d, "repeated_nested_message", 0, 8, 0); + ExpectLocation(&tree, d, "repeated_nested_message", 1, 11, 0); + + // Check for fields not set. For an invalid field, the location returned + // should be -1, -1. + ExpectLocation(&tree, d, "repeated_int64", 0, -1, -1); + ExpectLocation(&tree, d, "repeated_int32", 6, -1, -1); + ExpectLocation(&tree, d, "some_unknown_field", -1, -1, -1); + + // Verify inside the nested message. + const FieldDescriptor* nested_field = + d->FindFieldByName("optional_nested_message"); + + TextFormat::ParseInfoTree* nested_tree = + tree.GetTreeForNested(nested_field, -1); + ExpectLocation(nested_tree, nested_field->message_type(), "bb", -1, 6, 2); + + // Verify inside another nested message. + nested_field = d->FindFieldByName("repeated_nested_message"); + nested_tree = tree.GetTreeForNested(nested_field, 0); + ExpectLocation(nested_tree, nested_field->message_type(), "bb", -1, 9, 2); + + nested_tree = tree.GetTreeForNested(nested_field, 1); + ExpectLocation(nested_tree, nested_field->message_type(), "bb", -1, 12, 2); + + // Verify a NULL tree for an unknown nested field. + TextFormat::ParseInfoTree* unknown_nested_tree = + tree.GetTreeForNested(nested_field, 2); + + EXPECT_EQ(NULL, unknown_nested_tree); +} + TEST_F(TextFormatParserTest, ParseFieldValueFromString) { scoped_ptr<unittest::TestAllTypes> message(new unittest::TestAllTypes); const Descriptor* d = message->GetDescriptor(); @@ -921,10 +1020,10 @@ TEST_F(TextFormatParserTest, InvalidFieldValues) { 1, 16); ExpectFailure("optional_int32: 0x80000000\n", "Integer out of range.", 1, 17); - ExpectFailure("optional_int32: -0x80000001\n", - "Integer out of range.", 1, 18); ExpectFailure("optional_int64: 0x8000000000000000\n", "Integer out of range.", 1, 17); + ExpectFailure("optional_int32: -0x80000001\n", + "Integer out of range.", 1, 18); ExpectFailure("optional_int64: -0x8000000000000001\n", "Integer out of range.", 1, 18); @@ -1136,6 +1235,7 @@ TEST_F(TextFormatMessageSetTest, Deserialize) { EXPECT_EQ(2, descriptors.size()); } + } // namespace text_format_unittest } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/unittest.proto b/third_party/protobuf/src/google/protobuf/unittest.proto index 97ec674..0305be6 100644 --- a/third_party/protobuf/src/google/protobuf/unittest.proto +++ b/third_party/protobuf/src/google/protobuf/unittest.proto @@ -103,6 +103,12 @@ message TestAllTypes { optional string optional_string_piece = 24 [ctype=STRING_PIECE]; optional string optional_cord = 25 [ctype=CORD]; + // Defined in unittest_import_public.proto + optional protobuf_unittest_import.PublicImportMessage + optional_public_import_message = 26; + + optional NestedMessage optional_lazy_message = 27 [lazy=true]; + // Repeated repeated int32 repeated_int32 = 31; repeated int64 repeated_int64 = 32; @@ -135,6 +141,8 @@ message TestAllTypes { repeated string repeated_string_piece = 54 [ctype=STRING_PIECE]; repeated string repeated_cord = 55 [ctype=CORD]; + repeated NestedMessage repeated_lazy_message = 57 [lazy=true]; + // Singular with defaults optional int32 default_int32 = 61 [default = 41 ]; optional int64 default_int64 = 62 [default = 42 ]; @@ -216,6 +224,12 @@ extend TestAllExtensions { optional string optional_string_piece_extension = 24 [ctype=STRING_PIECE]; optional string optional_cord_extension = 25 [ctype=CORD]; + optional protobuf_unittest_import.PublicImportMessage + optional_public_import_message_extension = 26; + + optional TestAllTypes.NestedMessage + optional_lazy_message_extension = 27 [lazy=true]; + // Repeated repeated int32 repeated_int32_extension = 31; repeated int64 repeated_int64_extension = 32; @@ -250,6 +264,9 @@ extend TestAllExtensions { repeated string repeated_string_piece_extension = 54 [ctype=STRING_PIECE]; repeated string repeated_cord_extension = 55 [ctype=CORD]; + repeated TestAllTypes.NestedMessage + repeated_lazy_message_extension = 57 [lazy=true]; + // Singular with defaults optional int32 default_int32_extension = 61 [default = 41 ]; optional int64 default_int64_extension = 62 [default = 42 ]; @@ -397,6 +414,13 @@ message TestDupFieldNumber { // NO_PROTO1 optional group Bar = 3 { optional int32 a = 1; } // NO_PROTO1 } // NO_PROTO1 +// Additional messages for testing lazy fields. +message TestEagerMessage { + optional TestAllTypes sub_message = 1 [lazy=false]; +} +message TestLazyMessage { + optional TestAllTypes sub_message = 1 [lazy=true]; +} // Needed for a Python test. message TestNestedMessageHasBits { @@ -470,6 +494,8 @@ message TestExtremeDefaultValues { optional uint64 large_uint64 = 3 [default = 0xFFFFFFFFFFFFFFFF]; optional int32 small_int32 = 4 [default = -0x7FFFFFFF]; optional int64 small_int64 = 5 [default = -0x7FFFFFFFFFFFFFFF]; + optional int32 really_small_int32 = 21 [default = -0x80000000]; + optional int64 really_small_int64 = 22 [default = -0x8000000000000000]; // The default value here is UTF-8 for "\u1234". (We could also just type // the UTF-8 text directly into this text file rather than escape it, but @@ -500,6 +526,14 @@ message TestExtremeDefaultValues { // Note that in .proto file, "\?" is a valid way to escape ? in string // literals. optional string cpp_trigraph = 20 [default = "? \? ?? \?? \??? ??/ ?\?-"]; + + // String defaults containing the character '\000' + optional string string_with_zero = 23 [default = "hel\000lo"]; + optional bytes bytes_with_zero = 24 [default = "wor\000ld"]; + optional string string_piece_with_zero = 25 [ctype=STRING_PIECE, + default="ab\000c"]; + optional string cord_with_zero = 26 [ctype=CORD, + default="12\0003"]; } message SparseEnumMessage { @@ -511,10 +545,19 @@ message OneString { optional string data = 1; } +message MoreString { + repeated string data = 1; +} + message OneBytes { optional bytes data = 1; } +message MoreBytes { + repeated bytes data = 1; +} + + // Test messages for packed fields message TestPackedTypes { @@ -615,11 +658,56 @@ message TestRepeatedScalarDifferentTagSizes { repeated uint64 repeated_uint64 = 262143; } +// Test that if an optional or required message/group field appears multiple +// times in the input, they need to be merged. +message TestParsingMerge { + // RepeatedFieldsGenerator defines matching field types as TestParsingMerge, + // except that all fields are repeated. In the tests, we will serialize the + // RepeatedFieldsGenerator to bytes, and parse the bytes to TestParsingMerge. + // Repeated fields in RepeatedFieldsGenerator are expected to be merged into + // the corresponding required/optional fields in TestParsingMerge. + message RepeatedFieldsGenerator { + repeated TestAllTypes field1 = 1; + repeated TestAllTypes field2 = 2; + repeated TestAllTypes field3 = 3; + repeated group Group1 = 10 { + optional TestAllTypes field1 = 11; + } + repeated group Group2 = 20 { + optional TestAllTypes field1 = 21; + } + repeated TestAllTypes ext1 = 1000; + repeated TestAllTypes ext2 = 1001; + } + required TestAllTypes required_all_types = 1; + optional TestAllTypes optional_all_types = 2; + repeated TestAllTypes repeated_all_types = 3; + optional group OptionalGroup = 10 { + optional TestAllTypes optional_group_all_types = 11; + } + repeated group RepeatedGroup = 20 { + optional TestAllTypes repeated_group_all_types = 21; + } + extensions 1000 to max; + extend TestParsingMerge { + optional TestAllTypes optional_ext = 1000; + repeated TestAllTypes repeated_ext = 1001; + } +} + +message TestCommentInjectionMessage { + // */ <- This should not close the generated doc comment + optional string a = 1 [default="*/ <- Neither should this."]; +} + // Test that RPC services work. message FooRequest {} message FooResponse {} +message FooClientMessage {} +message FooServerMessage{} + service TestService { rpc Foo(FooRequest) returns (FooResponse); rpc Bar(BarRequest) returns (BarResponse); diff --git a/third_party/protobuf/src/google/protobuf/unittest_custom_options.proto b/third_party/protobuf/src/google/protobuf/unittest_custom_options.proto index a610785..e591d29 100644 --- a/third_party/protobuf/src/google/protobuf/unittest_custom_options.proto +++ b/third_party/protobuf/src/google/protobuf/unittest_custom_options.proto @@ -116,6 +116,12 @@ message CustomOptionFooRequest { message CustomOptionFooResponse { } +message CustomOptionFooClientMessage { +} + +message CustomOptionFooServerMessage { +} + service TestServiceWithCustomOptions { option (service_opt1) = -9876543210; @@ -364,3 +370,18 @@ enum AggregateEnum { option (enumopt) = { s:'EnumAnnotation' }; VALUE = 1 [(enumvalopt) = { s:'EnumValueAnnotation' }]; } + +// Test custom options for nested type. +message NestedOptionType { + message NestedMessage { + option (message_opt1) = 1001; + optional int32 nested_field = 1 [(field_opt1) = 1002]; + } + enum NestedEnum { + option (enum_opt1) = 1003; + NESTED_ENUM_VALUE = 1 [(enum_value_opt1) = 1004]; + } + extend google.protobuf.FileOptions { + optional int32 nested_extension = 7912573 [(field_opt2) = 1005]; + } +} diff --git a/third_party/protobuf/src/google/protobuf/unittest_import.proto b/third_party/protobuf/src/google/protobuf/unittest_import.proto index cd533ec..c115b11 100644 --- a/third_party/protobuf/src/google/protobuf/unittest_import.proto +++ b/third_party/protobuf/src/google/protobuf/unittest_import.proto @@ -49,6 +49,9 @@ option java_package = "com.google.protobuf.test"; // Do not set a java_outer_classname here to verify that Proto2 works without // one. +// Test public import +import public "google/protobuf/unittest_import_public.proto"; + message ImportMessage { optional int32 d = 1; } diff --git a/third_party/protobuf/src/google/protobuf/unittest_import_lite.proto b/third_party/protobuf/src/google/protobuf/unittest_import_lite.proto index ebaab5c..81b117f 100644 --- a/third_party/protobuf/src/google/protobuf/unittest_import_lite.proto +++ b/third_party/protobuf/src/google/protobuf/unittest_import_lite.proto @@ -38,6 +38,8 @@ option optimize_for = LITE_RUNTIME; option java_package = "com.google.protobuf"; +import public "google/protobuf/unittest_import_public_lite.proto"; + message ImportMessageLite { optional int32 d = 1; } diff --git a/third_party/protobuf/src/google/protobuf/unittest_import_public.proto b/third_party/protobuf/src/google/protobuf/unittest_import_public.proto new file mode 100644 index 0000000..ea5d1b1 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/unittest_import_public.proto @@ -0,0 +1,40 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: liujisi@google.com (Pherl Liu) + + +package protobuf_unittest_import; + +option java_package = "com.google.protobuf.test"; + +message PublicImportMessage { + optional int32 e = 1; +} diff --git a/third_party/protobuf/src/google/protobuf/unittest_import_public_lite.proto b/third_party/protobuf/src/google/protobuf/unittest_import_public_lite.proto new file mode 100644 index 0000000..d077563 --- /dev/null +++ b/third_party/protobuf/src/google/protobuf/unittest_import_public_lite.proto @@ -0,0 +1,42 @@ +// Protocol Buffers - Google's data interchange format +// Copyright 2008 Google Inc. All rights reserved. +// http://code.google.com/p/protobuf/ +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +// Author: liujisi@google.com (Pherl Liu) + + +package protobuf_unittest_import; + +option optimize_for = LITE_RUNTIME; + +option java_package = "com.google.protobuf"; + +message PublicImportMessageLite { + optional int32 e = 1; +} diff --git a/third_party/protobuf/src/google/protobuf/unittest_lite.proto b/third_party/protobuf/src/google/protobuf/unittest_lite.proto index cca6b49..a1764aa 100644 --- a/third_party/protobuf/src/google/protobuf/unittest_lite.proto +++ b/third_party/protobuf/src/google/protobuf/unittest_lite.proto @@ -85,6 +85,12 @@ message TestAllTypesLite { optional string optional_string_piece = 24 [ctype=STRING_PIECE]; optional string optional_cord = 25 [ctype=CORD]; + // Defined in unittest_import_public.proto + optional protobuf_unittest_import.PublicImportMessageLite + optional_public_import_message = 26; + + optional NestedMessage optional_lazy_message = 27 [lazy=true]; + // Repeated repeated int32 repeated_int32 = 31; repeated int64 repeated_int64 = 32; @@ -118,6 +124,8 @@ message TestAllTypesLite { repeated string repeated_string_piece = 54 [ctype=STRING_PIECE]; repeated string repeated_cord = 55 [ctype=CORD]; + repeated NestedMessage repeated_lazy_message = 57 [lazy=true]; + // Singular with defaults optional int32 default_int32 = 61 [default = 41 ]; optional int64 default_int64 = 62 [default = 42 ]; @@ -213,6 +221,12 @@ extend TestAllExtensionsLite { [ctype=STRING_PIECE]; optional string optional_cord_extension_lite = 25 [ctype=CORD]; + optional protobuf_unittest_import.PublicImportMessageLite + optional_public_import_message_extension_lite = 26; + + optional TestAllTypesLite.NestedMessage + optional_lazy_message_extension_lite = 27 [lazy=true]; + // Repeated repeated int32 repeated_int32_extension_lite = 31; repeated int64 repeated_int64_extension_lite = 32; @@ -249,6 +263,9 @@ extend TestAllExtensionsLite { [ctype=STRING_PIECE]; repeated string repeated_cord_extension_lite = 55 [ctype=CORD]; + repeated TestAllTypesLite.NestedMessage + repeated_lazy_message_extension_lite = 57 [lazy=true]; + // Singular with defaults optional int32 default_int32_extension_lite = 61 [default = 41 ]; optional int64 default_int64_extension_lite = 62 [default = 42 ]; @@ -310,3 +327,34 @@ message TestNestedExtensionLite { message TestDeprecatedLite { optional int32 deprecated_field = 1 [deprecated = true]; } + +// See the comments of the same type in unittest.proto. +message TestParsingMergeLite { + message RepeatedFieldsGenerator { + repeated TestAllTypesLite field1 = 1; + repeated TestAllTypesLite field2 = 2; + repeated TestAllTypesLite field3 = 3; + repeated group Group1 = 10 { + optional TestAllTypesLite field1 = 11; + } + repeated group Group2 = 20 { + optional TestAllTypesLite field1 = 21; + } + repeated TestAllTypesLite ext1 = 1000; + repeated TestAllTypesLite ext2 = 1001; + } + required TestAllTypesLite required_all_types = 1; + optional TestAllTypesLite optional_all_types = 2; + repeated TestAllTypesLite repeated_all_types = 3; + optional group OptionalGroup = 10 { + optional TestAllTypesLite optional_group_all_types = 11; + } + repeated group RepeatedGroup = 20 { + optional TestAllTypesLite repeated_group_all_types = 21; + } + extensions 1000 to max; + extend TestParsingMergeLite { + optional TestAllTypesLite optional_ext = 1000; + repeated TestAllTypesLite repeated_ext = 1001; + } +} diff --git a/third_party/protobuf/src/google/protobuf/unknown_field_set.cc b/third_party/protobuf/src/google/protobuf/unknown_field_set.cc index 38a36d5..b6aaa18 100644 --- a/third_party/protobuf/src/google/protobuf/unknown_field_set.cc +++ b/third_party/protobuf/src/google/protobuf/unknown_field_set.cc @@ -32,13 +32,14 @@ // Based on original Protocol Buffers design by // Sanjay Ghemawat, Jeff Dean, and others. -#include <google/protobuf/stubs/common.h> #include <google/protobuf/unknown_field_set.h> -#include <google/protobuf/stubs/stl_util-inl.h> + +#include <google/protobuf/stubs/common.h> #include <google/protobuf/io/coded_stream.h> #include <google/protobuf/io/zero_copy_stream.h> #include <google/protobuf/io/zero_copy_stream_impl_lite.h> #include <google/protobuf/wire_format_lite.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -75,6 +76,14 @@ void UnknownFieldSet::ClearFallback() { fields_->clear(); } +void UnknownFieldSet::ClearAndFreeMemory() { + if (fields_ != NULL) { + Clear(); + delete fields_; + fields_ = NULL; + } +} + void UnknownFieldSet::MergeFrom(const UnknownFieldSet& other) { for (int i = 0; i < other.field_count(); i++) { AddField(other.field(i)); @@ -89,8 +98,9 @@ int UnknownFieldSet::SpaceUsedExcludingSelf() const { const UnknownField& field = (*fields_)[i]; switch (field.type()) { case UnknownField::TYPE_LENGTH_DELIMITED: - total_size += sizeof(*field.length_delimited_) + - internal::StringSpaceUsedExcludingSelf(*field.length_delimited_); + total_size += sizeof(*field.length_delimited_.string_value_) + + internal::StringSpaceUsedExcludingSelf( + *field.length_delimited_.string_value_); break; case UnknownField::TYPE_GROUP: total_size += field.group_->SpaceUsed(); @@ -138,11 +148,12 @@ string* UnknownFieldSet::AddLengthDelimited(int number) { UnknownField field; field.number_ = number; field.type_ = UnknownField::TYPE_LENGTH_DELIMITED; - field.length_delimited_ = new string; + field.length_delimited_.string_value_ = new string; fields_->push_back(field); - return field.length_delimited_; + return field.length_delimited_.string_value_; } + UnknownFieldSet* UnknownFieldSet::AddGroup(int number) { if (fields_ == NULL) fields_ = new vector<UnknownField>; UnknownField field; @@ -159,6 +170,39 @@ void UnknownFieldSet::AddField(const UnknownField& field) { fields_->back().DeepCopy(); } +void UnknownFieldSet::DeleteSubrange(int start, int num) { + GOOGLE_DCHECK(fields_ != NULL); + // Delete the specified fields. + for (int i = 0; i < num; ++i) { + (*fields_)[i + start].Delete(); + } + // Slide down the remaining fields. + for (int i = start + num; i < fields_->size(); ++i) { + (*fields_)[i - num] = (*fields_)[i]; + } + // Pop off the # of deleted fields. + for (int i = 0; i < num; ++i) { + fields_->pop_back(); + } +} + +void UnknownFieldSet::DeleteByNumber(int number) { + if (fields_ == NULL) return; + int left = 0; // The number of fields left after deletion. + for (int i = 0; i < fields_->size(); ++i) { + UnknownField* field = &(*fields_)[i]; + if (field->number() == number) { + field->Delete(); + } else { + if (i != left) { + (*fields_)[left] = (*fields_)[i]; + } + ++left; + } + } + fields_->resize(left); +} + bool UnknownFieldSet::MergeFromCodedStream(io::CodedInputStream* input) { UnknownFieldSet other; @@ -190,7 +234,7 @@ bool UnknownFieldSet::ParseFromArray(const void* data, int size) { void UnknownField::Delete() { switch (type()) { case UnknownField::TYPE_LENGTH_DELIMITED: - delete length_delimited_; + delete length_delimited_.string_value_; break; case UnknownField::TYPE_GROUP: delete group_; @@ -203,7 +247,8 @@ void UnknownField::Delete() { void UnknownField::DeepCopy() { switch (type()) { case UnknownField::TYPE_LENGTH_DELIMITED: - length_delimited_ = new string(*length_delimited_); + length_delimited_.string_value_ = new string( + *length_delimited_.string_value_); break; case UnknownField::TYPE_GROUP: { UnknownFieldSet* group = new UnknownFieldSet; @@ -216,5 +261,22 @@ void UnknownField::DeepCopy() { } } + +void UnknownField::SerializeLengthDelimitedNoTag( + io::CodedOutputStream* output) const { + GOOGLE_DCHECK_EQ(TYPE_LENGTH_DELIMITED, type_); + const string& data = *length_delimited_.string_value_; + output->WriteVarint32(data.size()); + output->WriteString(data); +} + +uint8* UnknownField::SerializeLengthDelimitedNoTagToArray(uint8* target) const { + GOOGLE_DCHECK_EQ(TYPE_LENGTH_DELIMITED, type_); + const string& data = *length_delimited_.string_value_; + target = io::CodedOutputStream::WriteVarint32ToArray(data.size(), target); + target = io::CodedOutputStream::WriteStringToArray(data, target); + return target; +} + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/unknown_field_set.h b/third_party/protobuf/src/google/protobuf/unknown_field_set.h index 84c2e2b..825bba8 100644 --- a/third_party/protobuf/src/google/protobuf/unknown_field_set.h +++ b/third_party/protobuf/src/google/protobuf/unknown_field_set.h @@ -38,12 +38,24 @@ #ifndef GOOGLE_PROTOBUF_UNKNOWN_FIELD_SET_H__ #define GOOGLE_PROTOBUF_UNKNOWN_FIELD_SET_H__ +#include <assert.h> #include <string> #include <vector> -#include <google/protobuf/repeated_field.h> +#include <google/protobuf/stubs/common.h> +// TODO(jasonh): some people seem to rely on protobufs to include this for them! namespace google { namespace protobuf { + namespace io { + class CodedInputStream; // coded_stream.h + class CodedOutputStream; // coded_stream.h + class ZeroCopyInputStream; // zero_copy_stream.h + } + namespace internal { + class WireFormat; // wire_format.h + class UnknownFieldSetFieldSkipperUsingCord; + // extension_set_heavy.cc + } class Message; // message.h class UnknownField; // below @@ -68,6 +80,9 @@ class LIBPROTOBUF_EXPORT UnknownFieldSet { // Remove all fields. inline void Clear(); + // Remove all fields and deallocate internal data objects + void ClearAndFreeMemory(); + // Is this set empty? inline bool empty() const; @@ -107,6 +122,15 @@ class LIBPROTOBUF_EXPORT UnknownFieldSet { // Adds an unknown field from another set. void AddField(const UnknownField& field); + // Delete fields with indices in the range [start .. start+num-1]. + // Caution: implementation moves all fields with indices [start+num .. ]. + void DeleteSubrange(int start, int num); + + // Delete all fields with a specific field number. The order of left fields + // is preserved. + // Caution: implementation moves all fields after the first deleted field. + void DeleteByNumber(int number); + // Parsing helpers ------------------------------------------------- // These work exactly like the similarly-named methods of Message. @@ -119,6 +143,7 @@ class LIBPROTOBUF_EXPORT UnknownFieldSet { } private: + void ClearFallback(); vector<UnknownField>* fields_; @@ -159,6 +184,15 @@ class LIBPROTOBUF_EXPORT UnknownField { inline string* mutable_length_delimited(); inline UnknownFieldSet* mutable_group(); + // Serialization API. + // These methods can take advantage of the underlying implementation and may + // archieve a better performance than using getters to retrieve the data and + // do the serialization yourself. + void SerializeLengthDelimitedNoTag(io::CodedOutputStream* output) const; + uint8* SerializeLengthDelimitedNoTagToArray(uint8* target) const; + + inline int GetLengthDelimitedSize() const; + private: friend class UnknownFieldSet; @@ -168,13 +202,16 @@ class LIBPROTOBUF_EXPORT UnknownField { // Make a deep copy of any pointers in this UnknownField. void DeepCopy(); + unsigned int number_ : 29; unsigned int type_ : 3; union { uint64 varint_; uint32 fixed32_; uint64 fixed64_; - string* length_delimited_; + mutable union { + string* string_value_; + } length_delimited_; UnknownFieldSet* group_; }; }; @@ -211,57 +248,63 @@ inline void UnknownFieldSet::AddLengthDelimited( AddLengthDelimited(number)->assign(value); } + inline int UnknownField::number() const { return number_; } inline UnknownField::Type UnknownField::type() const { return static_cast<Type>(type_); } inline uint64 UnknownField::varint () const { - GOOGLE_DCHECK_EQ(type_, TYPE_VARINT); + assert(type_ == TYPE_VARINT); return varint_; } inline uint32 UnknownField::fixed32() const { - GOOGLE_DCHECK_EQ(type_, TYPE_FIXED32); + assert(type_ == TYPE_FIXED32); return fixed32_; } inline uint64 UnknownField::fixed64() const { - GOOGLE_DCHECK_EQ(type_, TYPE_FIXED64); + assert(type_ == TYPE_FIXED64); return fixed64_; } inline const string& UnknownField::length_delimited() const { - GOOGLE_DCHECK_EQ(type_, TYPE_LENGTH_DELIMITED); - return *length_delimited_; + assert(type_ == TYPE_LENGTH_DELIMITED); + return *length_delimited_.string_value_; } inline const UnknownFieldSet& UnknownField::group() const { - GOOGLE_DCHECK_EQ(type_, TYPE_GROUP); + assert(type_ == TYPE_GROUP); return *group_; } inline void UnknownField::set_varint(uint64 value) { - GOOGLE_DCHECK_EQ(type_, TYPE_VARINT); + assert(type_ == TYPE_VARINT); varint_ = value; } inline void UnknownField::set_fixed32(uint32 value) { - GOOGLE_DCHECK_EQ(type_, TYPE_FIXED32); + assert(type_ == TYPE_FIXED32); fixed32_ = value; } inline void UnknownField::set_fixed64(uint64 value) { - GOOGLE_DCHECK_EQ(type_, TYPE_FIXED64); + assert(type_ == TYPE_FIXED64); fixed64_ = value; } inline void UnknownField::set_length_delimited(const string& value) { - GOOGLE_DCHECK_EQ(type_, TYPE_LENGTH_DELIMITED); - length_delimited_->assign(value); + assert(type_ == TYPE_LENGTH_DELIMITED); + length_delimited_.string_value_->assign(value); } inline string* UnknownField::mutable_length_delimited() { - GOOGLE_DCHECK_EQ(type_, TYPE_LENGTH_DELIMITED); - return length_delimited_; + assert(type_ == TYPE_LENGTH_DELIMITED); + return length_delimited_.string_value_; } inline UnknownFieldSet* UnknownField::mutable_group() { - GOOGLE_DCHECK_EQ(type_, TYPE_GROUP); + assert(type_ == TYPE_GROUP); return group_; } +inline int UnknownField::GetLengthDelimitedSize() const { + GOOGLE_DCHECK_EQ(TYPE_LENGTH_DELIMITED, type_); + return length_delimited_.string_value_->size(); +} + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/unknown_field_set_unittest.cc b/third_party/protobuf/src/google/protobuf/unknown_field_set_unittest.cc index 1235c9e..c6b8769 100644 --- a/third_party/protobuf/src/google/protobuf/unknown_field_set_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/unknown_field_set_unittest.cc @@ -46,15 +46,13 @@ #include <google/protobuf/stubs/common.h> #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { using internal::WireFormat; -namespace { - class UnknownFieldSetTest : public testing::Test { protected: virtual void SetUp() { @@ -107,6 +105,8 @@ class UnknownFieldSetTest : public testing::Test { UnknownFieldSet* unknown_fields_; }; +namespace { + TEST_F(UnknownFieldSetTest, AllFieldsPresent) { // All fields of TestAllTypes should be present, in numeric order (because // that's the order we parsed them in). Fields that are not valid field @@ -297,12 +297,21 @@ TEST_F(UnknownFieldSetTest, MergeFrom) { destination.DebugString()); } + TEST_F(UnknownFieldSetTest, Clear) { // Clear the set. empty_message_.Clear(); EXPECT_EQ(0, unknown_fields_->field_count()); } +TEST_F(UnknownFieldSetTest, ClearAndFreeMemory) { + EXPECT_GT(unknown_fields_->field_count(), 0); + unknown_fields_->ClearAndFreeMemory(); + EXPECT_EQ(0, unknown_fields_->field_count()); + unknown_fields_->AddVarint(123456, 654321); + EXPECT_EQ(1, unknown_fields_->field_count()); +} + TEST_F(UnknownFieldSetTest, ParseKnownAndUnknown) { // Test mixing known and unknown fields when parsing. @@ -498,6 +507,7 @@ TEST_F(UnknownFieldSetTest, SpaceUsed) { EXPECT_LT(base_size, empty_message.SpaceUsed()); } + TEST_F(UnknownFieldSetTest, Empty) { UnknownFieldSet unknown_fields; EXPECT_TRUE(unknown_fields.empty()); @@ -507,6 +517,78 @@ TEST_F(UnknownFieldSetTest, Empty) { EXPECT_TRUE(unknown_fields.empty()); } +TEST_F(UnknownFieldSetTest, DeleteSubrange) { + // Exhaustively test the deletion of every possible subrange in arrays of all + // sizes from 0 through 9. + for (int size = 0; size < 10; ++size) { + for (int num = 0; num <= size; ++num) { + for (int start = 0; start < size - num; ++start) { + // Create a set with "size" fields. + UnknownFieldSet unknown; + for (int i = 0; i < size; ++i) { + unknown.AddFixed32(i, i); + } + // Delete the specified subrange. + unknown.DeleteSubrange(start, num); + // Make sure the resulting field values are still correct. + EXPECT_EQ(size - num, unknown.field_count()); + for (int i = 0; i < unknown.field_count(); ++i) { + if (i < start) { + EXPECT_EQ(i, unknown.field(i).fixed32()); + } else { + EXPECT_EQ(i + num, unknown.field(i).fixed32()); + } + } + } + } + } +} + +void CheckDeleteByNumber(const vector<int>& field_numbers, int deleted_number, + const vector<int>& expected_field_nubmers) { + UnknownFieldSet unknown_fields; + for (int i = 0; i < field_numbers.size(); ++i) { + unknown_fields.AddFixed32(field_numbers[i], i); + } + unknown_fields.DeleteByNumber(deleted_number); + ASSERT_EQ(expected_field_nubmers.size(), unknown_fields.field_count()); + for (int i = 0; i < expected_field_nubmers.size(); ++i) { + EXPECT_EQ(expected_field_nubmers[i], + unknown_fields.field(i).number()); + } +} + +#define MAKE_VECTOR(x) vector<int>(x, x + GOOGLE_ARRAYSIZE(x)) +TEST_F(UnknownFieldSetTest, DeleteByNumber) { + CheckDeleteByNumber(vector<int>(), 1, vector<int>()); + static const int kTestFieldNumbers1[] = {1, 2, 3}; + static const int kFieldNumberToDelete1 = 1; + static const int kExpectedFieldNumbers1[] = {2, 3}; + CheckDeleteByNumber(MAKE_VECTOR(kTestFieldNumbers1), kFieldNumberToDelete1, + MAKE_VECTOR(kExpectedFieldNumbers1)); + static const int kTestFieldNumbers2[] = {1, 2, 3}; + static const int kFieldNumberToDelete2 = 2; + static const int kExpectedFieldNumbers2[] = {1, 3}; + CheckDeleteByNumber(MAKE_VECTOR(kTestFieldNumbers2), kFieldNumberToDelete2, + MAKE_VECTOR(kExpectedFieldNumbers2)); + static const int kTestFieldNumbers3[] = {1, 2, 3}; + static const int kFieldNumberToDelete3 = 3; + static const int kExpectedFieldNumbers3[] = {1, 2}; + CheckDeleteByNumber(MAKE_VECTOR(kTestFieldNumbers3), kFieldNumberToDelete3, + MAKE_VECTOR(kExpectedFieldNumbers3)); + static const int kTestFieldNumbers4[] = {1, 2, 1, 4, 1}; + static const int kFieldNumberToDelete4 = 1; + static const int kExpectedFieldNumbers4[] = {2, 4}; + CheckDeleteByNumber(MAKE_VECTOR(kTestFieldNumbers4), kFieldNumberToDelete4, + MAKE_VECTOR(kExpectedFieldNumbers4)); + static const int kTestFieldNumbers5[] = {1, 2, 3, 4, 5}; + static const int kFieldNumberToDelete5 = 6; + static const int kExpectedFieldNumbers5[] = {1, 2, 3, 4, 5}; + CheckDeleteByNumber(MAKE_VECTOR(kTestFieldNumbers5), kFieldNumberToDelete5, + MAKE_VECTOR(kExpectedFieldNumbers5)); +} +#undef MAKE_VECTOR } // namespace + } // namespace protobuf } // namespace google diff --git a/third_party/protobuf/src/google/protobuf/wire_format.cc b/third_party/protobuf/src/google/protobuf/wire_format.cc index d07f633..491eede 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format.cc +++ b/third_party/protobuf/src/google/protobuf/wire_format.cc @@ -48,12 +48,11 @@ #include <google/protobuf/unknown_field_set.h> + namespace google { namespace protobuf { namespace internal { -using internal::WireFormatLite; - namespace { // This function turns out to be convenient when using some macros later. @@ -340,10 +339,7 @@ bool WireFormat::ParseAndMergeMessageSetItem( const FieldDescriptor* field = NULL; // If we see message data before the type_id, we'll append it to this so - // we can parse it later. This will probably never happen in practice, - // as no MessageSet encoder I know of writes the message before the type ID. - // But, it's technically valid so we should allow it. - // TODO(kenton): Use a Cord instead? Do I care? + // we can parse it later. string message_data; while (true) { @@ -381,7 +377,10 @@ bool WireFormat::ParseAndMergeMessageSetItem( uint32 length; if (!input->ReadVarint32(&length)) return false; if (!input->ReadString(&temp, length)) return false; - message_data.append(temp); + io::StringOutputStream output_stream(&message_data); + io::CodedOutputStream coded_output(&output_stream); + coded_output.WriteVarint32(length); + coded_output.WriteString(temp); } else { // Already saw type_id, so we can parse this directly. if (!ParseAndMergeField(fake_tag, field, message, input)) { @@ -754,10 +753,10 @@ void WireFormat::VerifyUTF8StringFallback(const char* data, break; // no default case: have the compiler warn if a case is not covered. } - GOOGLE_LOG(ERROR) << "Encountered string containing invalid UTF-8 data while " + GOOGLE_LOG(ERROR) << "String field contains invalid UTF-8 data when " << operation_str - << " protocol buffer. Strings must contain only UTF-8; " - "use the 'bytes' type for raw bytes."; + << " a protocol buffer. Use the 'bytes' type if you intend to " + "send raw bytes."; } } diff --git a/third_party/protobuf/src/google/protobuf/wire_format.h b/third_party/protobuf/src/google/protobuf/wire_format.h index d52bc4a..8a89881 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format.h +++ b/third_party/protobuf/src/google/protobuf/wire_format.h @@ -40,6 +40,7 @@ #define GOOGLE_PROTOBUF_WIRE_FORMAT_H__ #include <string> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/descriptor.pb.h> #include <google/protobuf/descriptor.h> #include <google/protobuf/message.h> diff --git a/third_party/protobuf/src/google/protobuf/wire_format_lite.cc b/third_party/protobuf/src/google/protobuf/wire_format_lite.cc index 3bf9268..6c21478 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format_lite.cc +++ b/third_party/protobuf/src/google/protobuf/wire_format_lite.cc @@ -40,7 +40,7 @@ #include <google/protobuf/stubs/common.h> #include <google/protobuf/io/coded_stream_inl.h> #include <google/protobuf/io/zero_copy_stream.h> -#include <google/protobuf/io/zero_copy_stream_impl.h> +#include <google/protobuf/io/zero_copy_stream_impl_lite.h> #include <google/protobuf/unknown_field_set.h> namespace google { @@ -506,12 +506,14 @@ void WireFormatLite::WriteString(int field_number, const string& value, io::CodedOutputStream* output) { // String is for UTF-8 text only WriteTag(field_number, WIRETYPE_LENGTH_DELIMITED, output); + GOOGLE_CHECK(value.size() <= kint32max); output->WriteVarint32(value.size()); output->WriteString(value); } void WireFormatLite::WriteBytes(int field_number, const string& value, io::CodedOutputStream* output) { WriteTag(field_number, WIRETYPE_LENGTH_DELIMITED, output); + GOOGLE_CHECK(value.size() <= kint32max); output->WriteVarint32(value.size()); output->WriteString(value); } diff --git a/third_party/protobuf/src/google/protobuf/wire_format_lite.h b/third_party/protobuf/src/google/protobuf/wire_format_lite.h index 5f9f2a3..98d8004 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format_lite.h +++ b/third_party/protobuf/src/google/protobuf/wire_format_lite.h @@ -41,16 +41,14 @@ #define GOOGLE_PROTOBUF_WIRE_FORMAT_LITE_H__ #include <string> +#include <google/protobuf/stubs/common.h> #include <google/protobuf/message_lite.h> +#include <google/protobuf/io/coded_stream.h> // for CodedOutputStream::Varint32Size namespace google { namespace protobuf { template <typename T> class RepeatedField; // repeated_field.h - namespace io { - class CodedInputStream; // coded_stream.h - class CodedOutputStream; // coded_stream.h - } class UnknownFieldSet; } @@ -521,6 +519,10 @@ class LIBPROTOBUF_EXPORT WireFormatLite { template<typename MessageType> static inline int MessageSizeNoVirtual(const MessageType& value); + // Given the length of data, calculate the byte size of the data on the + // wire if we encode the data as a length delimited field. + static inline int LengthDelimitedSize(int length); + private: // A helper method for the repeated primitive reader. This method has // optimizations for primitive types that have fixed size on the wire, and @@ -556,7 +558,7 @@ class LIBPROTOBUF_EXPORT FieldSkipper { // saves it as an unknown varint. void SkipUnknownEnum(int field_number, int value); - private: + protected: UnknownFieldSet* unknown_fields_; }; diff --git a/third_party/protobuf/src/google/protobuf/wire_format_lite_inl.h b/third_party/protobuf/src/google/protobuf/wire_format_lite_inl.h index 103b0bd..0f97386 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format_lite_inl.h +++ b/third_party/protobuf/src/google/protobuf/wire_format_lite_inl.h @@ -41,7 +41,6 @@ #include <google/protobuf/message_lite.h> #include <google/protobuf/repeated_field.h> #include <google/protobuf/wire_format_lite.h> -#include <google/protobuf/generated_message_util.h> #include <google/protobuf/io/coded_stream.h> @@ -749,8 +748,7 @@ inline int WireFormatLite::GroupSize(const MessageLite& value) { return value.ByteSize(); } inline int WireFormatLite::MessageSize(const MessageLite& value) { - int size = value.ByteSize(); - return io::CodedOutputStream::VarintSize32(size) + size; + return LengthDelimitedSize(value.ByteSize()); } // See comment on ReadGroupNoVirtual to understand the need for this template @@ -763,8 +761,12 @@ inline int WireFormatLite::GroupSizeNoVirtual( template<typename MessageType_WorkAroundCppLookupDefect> inline int WireFormatLite::MessageSizeNoVirtual( const MessageType_WorkAroundCppLookupDefect& value) { - int size = value.MessageType_WorkAroundCppLookupDefect::ByteSize(); - return io::CodedOutputStream::VarintSize32(size) + size; + return LengthDelimitedSize( + value.MessageType_WorkAroundCppLookupDefect::ByteSize()); +} + +inline int WireFormatLite::LengthDelimitedSize(int length) { + return io::CodedOutputStream::VarintSize32(length) + length; } } // namespace internal diff --git a/third_party/protobuf/src/google/protobuf/wire_format_unittest.cc b/third_party/protobuf/src/google/protobuf/wire_format_unittest.cc index 867970c..9822828 100644 --- a/third_party/protobuf/src/google/protobuf/wire_format_unittest.cc +++ b/third_party/protobuf/src/google/protobuf/wire_format_unittest.cc @@ -44,7 +44,7 @@ #include <google/protobuf/stubs/common.h> #include <google/protobuf/testing/googletest.h> #include <gtest/gtest.h> -#include <google/protobuf/stubs/stl_util-inl.h> +#include <google/protobuf/stubs/stl_util.h> namespace google { namespace protobuf { @@ -480,6 +480,54 @@ TEST(WireFormatTest, ParseMessageSet) { EXPECT_EQ(message_set.DebugString(), dynamic_message_set.DebugString()); } +TEST(WireFormatTest, ParseMessageSetWithReverseTagOrder) { + string data; + { + unittest::TestMessageSetExtension1 message; + message.set_i(123); + // Build a MessageSet manually with its message content put before its + // type_id. + io::StringOutputStream output_stream(&data); + io::CodedOutputStream coded_output(&output_stream); + coded_output.WriteTag(WireFormatLite::kMessageSetItemStartTag); + // Write the message content first. + WireFormatLite::WriteTag(WireFormatLite::kMessageSetMessageNumber, + WireFormatLite::WIRETYPE_LENGTH_DELIMITED, + &coded_output); + coded_output.WriteVarint32(message.ByteSize()); + message.SerializeWithCachedSizes(&coded_output); + // Write the type id. + uint32 type_id = message.GetDescriptor()->extension(0)->number(); + WireFormatLite::WriteUInt32(WireFormatLite::kMessageSetTypeIdNumber, + type_id, &coded_output); + coded_output.WriteTag(WireFormatLite::kMessageSetItemEndTag); + } + { + unittest::TestMessageSet message_set; + ASSERT_TRUE(message_set.ParseFromString(data)); + + EXPECT_EQ(123, message_set.GetExtension( + unittest::TestMessageSetExtension1::message_set_extension).i()); + } + { + // Test parse the message via Reflection. + unittest::TestMessageSet message_set; + io::CodedInputStream input( + reinterpret_cast<const uint8*>(data.data()), data.size()); + EXPECT_TRUE(WireFormat::ParseAndMergePartial(&input, &message_set)); + EXPECT_TRUE(input.ConsumedEntireMessage()); + + EXPECT_EQ(123, message_set.GetExtension( + unittest::TestMessageSetExtension1::message_set_extension).i()); + } +} + +TEST(WireFormatTest, ParseBrokenMessageSet) { + unittest::TestMessageSet message_set; + string input("goodbye"); // Invalid wire format data. + EXPECT_FALSE(message_set.ParseFromString(input)); +} + TEST(WireFormatTest, RecursionLimit) { unittest::TestRecursiveMessage message; message.mutable_a()->mutable_a()->mutable_a()->mutable_a()->set_i(1); @@ -800,6 +848,10 @@ bool ReadMessage(const string &wire_buffer, T *message) { return message->ParseFromArray(wire_buffer.data(), wire_buffer.size()); } +bool StartsWith(const string& s, const string& prefix) { + return s.substr(0, prefix.length()) == prefix; +} + TEST(Utf8ValidationTest, WriteInvalidUTF8String) { string wire_buffer; protobuf_unittest::OneString input; @@ -811,11 +863,10 @@ TEST(Utf8ValidationTest, WriteInvalidUTF8String) { } #ifdef GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED ASSERT_EQ(1, errors.size()); - EXPECT_EQ("Encountered string containing invalid UTF-8 data while " - "serializing protocol buffer. Strings must contain only UTF-8; " - "use the 'bytes' type for raw bytes.", - errors[0]); - + EXPECT_TRUE(StartsWith(errors[0], + "String field contains invalid UTF-8 data when " + "serializing a protocol buffer. Use the " + "'bytes' type if you intend to send raw bytes.")); #else ASSERT_EQ(0, errors.size()); #endif // GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED @@ -834,10 +885,10 @@ TEST(Utf8ValidationTest, ReadInvalidUTF8String) { } #ifdef GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED ASSERT_EQ(1, errors.size()); - EXPECT_EQ("Encountered string containing invalid UTF-8 data while " - "parsing protocol buffer. Strings must contain only UTF-8; " - "use the 'bytes' type for raw bytes.", - errors[0]); + EXPECT_TRUE(StartsWith(errors[0], + "String field contains invalid UTF-8 data when " + "parsing a protocol buffer. Use the " + "'bytes' type if you intend to send raw bytes.")); #else ASSERT_EQ(0, errors.size()); @@ -899,6 +950,28 @@ TEST(Utf8ValidationTest, ReadArbitraryBytes) { EXPECT_EQ(input.data(), output.data()); } +TEST(Utf8ValidationTest, ParseRepeatedString) { + protobuf_unittest::MoreBytes input; + input.add_data(kValidUTF8String); + input.add_data(kInvalidUTF8String); + input.add_data(kInvalidUTF8String); + string wire_buffer = input.SerializeAsString(); + + protobuf_unittest::MoreString output; + vector<string> errors; + { + ScopedMemoryLog log; + ReadMessage(wire_buffer, &output); + errors = log.GetMessages(ERROR); + } +#ifdef GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED + ASSERT_EQ(2, errors.size()); +#else + ASSERT_EQ(0, errors.size()); +#endif // GOOGLE_PROTOBUF_UTF8_VALIDATION_ENABLED + EXPECT_EQ(wire_buffer, output.SerializeAsString()); +} + } // namespace } // namespace internal } // namespace protobuf diff --git a/third_party/protobuf/vsprojects/extract_includes.bat b/third_party/protobuf/vsprojects/extract_includes.bat index 70fdf73..4dcb4e1 100755 --- a/third_party/protobuf/vsprojects/extract_includes.bat +++ b/third_party/protobuf/vsprojects/extract_includes.bat @@ -17,6 +17,7 @@ copy ..\src\google\protobuf\descriptor.pb.h include\google\protobuf\descriptor.p copy ..\src\google\protobuf\descriptor_database.h include\google\protobuf\descriptor_database.h
copy ..\src\google\protobuf\dynamic_message.h include\google\protobuf\dynamic_message.h
copy ..\src\google\protobuf\extension_set.h include\google\protobuf\extension_set.h
+copy ..\src\google\protobuf\generated_enum_reflection.h include\google\protobuf\generated_enum_reflection.h
copy ..\src\google\protobuf\generated_message_util.h include\google\protobuf\generated_message_util.h
copy ..\src\google\protobuf\generated_message_reflection.h include\google\protobuf\generated_message_reflection.h
copy ..\src\google\protobuf\message.h include\google\protobuf\message.h
|