Stop doing unnecessary UTF-8 to UTF-16 conversions in JSONWriter.

The JSONReader only accepts UTF-8 input strings and converts \uXXXX sequences
back into UTF-8. However, the JSONWriter converts all non-ASCII characters to
UTF-16 escape sequences. This round-tripping is sub-optimal, as noted in a
TODO from r54359.

One reason for this may be that JsonDoubleQuote(), used by JSONWriter, does not
handle UTF-8 bytes correctly, interpreting them as code points and writing them
out as \u00XX sequences. If this were read back through a RFC-compliant JSON
parser, the result would be an invalid encoding error. JsonDoubleQuote() does
handle UTF-16 correctly, though.

This rewrites the base/json/string_escape.h API and fixes the above UTF-8 issue
by dividing callers up into three groups:

1. Those that pass valid UTF-8 to be escaped. Prior to this change, very few
   callers used this variant. Those that did were likely using ASCII, otherwise
   the output would be mangled due to the above issue. Now, valid UTF-8 will be
   passed through to the output unescaped. Invalid UTF-8 sequences are replaced
   with U+FFFD.

2. Those that pass valid UTF-16 to be escaped. This function now validates that
   the input is valid UTF-16, and then converts it to unescaped UTF-8 sequences
   for the output.

3. Those that pass arbitrary byte arrays as std::string and expect a non-RFC-
   compliant encoding of the binary data using \uXXXX escapes. This behavior is
   now in the EscapeBytesAsInvalidJSONString() function. It is only used by
   callers who want a "debug string" but do not expect to actually parse the
   output as valid JSON, since it is not.

Additionally, this removes the JSONWriter::OPTIONS_DO_NOT_ESCAPE flag, since
the writer can now handle UTF-8 appropriately.

BUG=15466

Review URL: https://codereview.chromium.org/100823007

git-svn-id: svn://svn.chromium.org/chrome/trunk/src@239800 0039d316-1c4b-4281-b951-d872f2087c98

2 files changed, 4 insertions, 5 deletions

diff --git a/sync/internal_api/public/base/ordinal.h b/sync/internal_api/public/base/ordinal.h
index c347b7b..cb67b45 100644
--- a/sync/internal_api/public/base/ordinal.h
+++ b/sync/internal_api/public/base/ordinal.h
@@ -260,8 +260,8 @@ bool Ordinal<Traits>::EqualsOrBothInvalid(const Ordinal& other) const {
 
 template <typename Traits>
 std::string Ordinal<Traits>::ToDebugString() const {
-  std::string debug_string;
-  base::JsonDoubleQuote(bytes_, false /* put_in_quotes */, &debug_string);
+  std::string debug_string =
+      base::EscapeBytesAsInvalidJSONString(bytes_, false /* put_in_quotes */);
   if (!is_valid_) {
     debug_string = "INVALID[" + debug_string + "]";
   }
diff --git a/sync/internal_api/public/base/progress_marker_map.cc b/sync/internal_api/public/base/progress_marker_map.cc
index b281013..ea1f177 100644
--- a/sync/internal_api/public/base/progress_marker_map.cc
+++ b/sync/internal_api/public/base/progress_marker_map.cc
@@ -16,9 +16,8 @@ scoped_ptr<base::DictionaryValue> ProgressMarkerMapToValue(
   for (ProgressMarkerMap::const_iterator it = marker_map.begin();
        it != marker_map.end(); ++it) {
     std::string printable_payload;
-    base::JsonDoubleQuote(it->second,
-                          false /* put_in_quotes */,
-                          &printable_payload);
+    base::EscapeJSONString(
+        it->second, false /* put_in_quotes */, &printable_payload);
     value->SetString(ModelTypeToString(it->first), printable_payload);
   }
   return value.Pass();