summaryrefslogtreecommitdiffstats
path: root/base/json/string_escape_unittest.cc
Commit message (Collapse)AuthorAgeFilesLines
* Stop doing unnecessary UTF-8 to UTF-16 conversions in JSONWriter.rsesek@chromium.org2013-12-111-53/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The JSONReader only accepts UTF-8 input strings and converts \uXXXX sequences back into UTF-8. However, the JSONWriter converts all non-ASCII characters to UTF-16 escape sequences. This round-tripping is sub-optimal, as noted in a TODO from r54359. One reason for this may be that JsonDoubleQuote(), used by JSONWriter, does not handle UTF-8 bytes correctly, interpreting them as code points and writing them out as \u00XX sequences. If this were read back through a RFC-compliant JSON parser, the result would be an invalid encoding error. JsonDoubleQuote() does handle UTF-16 correctly, though. This rewrites the base/json/string_escape.h API and fixes the above UTF-8 issue by dividing callers up into three groups: 1. Those that pass valid UTF-8 to be escaped. Prior to this change, very few callers used this variant. Those that did were likely using ASCII, otherwise the output would be mangled due to the above issue. Now, valid UTF-8 will be passed through to the output unescaped. Invalid UTF-8 sequences are replaced with U+FFFD. 2. Those that pass valid UTF-16 to be escaped. This function now validates that the input is valid UTF-16, and then converts it to unescaped UTF-8 sequences for the output. 3. Those that pass arbitrary byte arrays as std::string and expect a non-RFC- compliant encoding of the binary data using \uXXXX escapes. This behavior is now in the EscapeBytesAsInvalidJSONString() function. It is only used by callers who want a "debug string" but do not expect to actually parse the output as valid JSON, since it is not. Additionally, this removes the JSONWriter::OPTIONS_DO_NOT_ESCAPE flag, since the writer can now handle UTF-8 appropriately. BUG=15466 Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=239800 Reverted: https://src.chromium.org/viewvc/chrome?view=rev&revision=240082 R=asanka@chromium.org, bauerb@chromium.org, mark@chromium.org, thakis@chromium.org, zea@chromium.org Review URL: https://codereview.chromium.org/100823007 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@240190 0039d316-1c4b-4281-b951-d872f2087c98
* Revert of https://codereview.chromium.org/100823007/antrim@chromium.org2013-12-111-131/+53
| | | | | | | | | | | | Reason for revert: This patchset breaks at least displayment of Russian localized strings on Chromeos login screen. TBR=mark@chromium.org,jshin@chromium.org,thakis@chromium.org,asanka@chromium.org,zea@chromium.org,bauerb@chromium.org,rsesek@chromium.org NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/106793004 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@240082 0039d316-1c4b-4281-b951-d872f2087c98
* Stop doing unnecessary UTF-8 to UTF-16 conversions in JSONWriter.rsesek@chromium.org2013-12-101-53/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The JSONReader only accepts UTF-8 input strings and converts \uXXXX sequences back into UTF-8. However, the JSONWriter converts all non-ASCII characters to UTF-16 escape sequences. This round-tripping is sub-optimal, as noted in a TODO from r54359. One reason for this may be that JsonDoubleQuote(), used by JSONWriter, does not handle UTF-8 bytes correctly, interpreting them as code points and writing them out as \u00XX sequences. If this were read back through a RFC-compliant JSON parser, the result would be an invalid encoding error. JsonDoubleQuote() does handle UTF-16 correctly, though. This rewrites the base/json/string_escape.h API and fixes the above UTF-8 issue by dividing callers up into three groups: 1. Those that pass valid UTF-8 to be escaped. Prior to this change, very few callers used this variant. Those that did were likely using ASCII, otherwise the output would be mangled due to the above issue. Now, valid UTF-8 will be passed through to the output unescaped. Invalid UTF-8 sequences are replaced with U+FFFD. 2. Those that pass valid UTF-16 to be escaped. This function now validates that the input is valid UTF-16, and then converts it to unescaped UTF-8 sequences for the output. 3. Those that pass arbitrary byte arrays as std::string and expect a non-RFC- compliant encoding of the binary data using \uXXXX escapes. This behavior is now in the EscapeBytesAsInvalidJSONString() function. It is only used by callers who want a "debug string" but do not expect to actually parse the output as valid JSON, since it is not. Additionally, this removes the JSONWriter::OPTIONS_DO_NOT_ESCAPE flag, since the writer can now handle UTF-8 appropriately. BUG=15466 Review URL: https://codereview.chromium.org/100823007 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@239800 0039d316-1c4b-4281-b951-d872f2087c98
* Let JsonDoubleQuote accept StringPiecewangxianzhu@chromium.org2013-09-061-2/+6
| | | | | | | | | | | | So that the functions can escape 'const char*' or 'const char16*' without creating a std::string or string16 object for the parameter. BUG=none TEST=StringEscapeTest.JsonDoubleQuoteNarrow Review URL: https://chromiumcodereview.appspot.com/18648003 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@221625 0039d316-1c4b-4281-b951-d872f2087c98
* Use a direct include of utf_string_conversions.h in android_webview/, apps/, ↵avi@chromium.org2013-06-071-1/+1
| | | | | | | | | | | ash/, base/. BUG=none TEST=none Review URL: https://chromiumcodereview.appspot.com/15735027 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@204874 0039d316-1c4b-4281-b951-d872f2087c98
* Improve the underlying escaping function JsonDoubleQuoteT to escape < and > ↵inferno@chromium.org2010-04-061-0/+2
| | | | | | | | | | characters BY DEFAULT to prevent script execution. BUG=40147 TEST=StringEscapeTest.* Review URL: http://codereview.chromium.org/1512013 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@43695 0039d316-1c4b-4281-b951-d872f2087c98
* base: string_util.h -> utf_string_conversions.h fix.jhawkins@chromium.org2010-03-091-4/+3
| | | | | | | | BUG=none TEST=none Review URL: http://codereview.chromium.org/678001 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@40972 0039d316-1c4b-4281-b951-d872f2087c98
* Move the json-related files into a separate json directory. This hopefully alsobrettw@chromium.org2009-10-231-0/+99
makes the naming of string_escape more clear (it's actually JSON-specific). Move the files into the base namespace. TEST=none BUG=none Review URL: http://codereview.chromium.org/316016 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@29934 0039d316-1c4b-4281-b951-d872f2087c98
generated by cgit v1.1 at 2025-03-01 05:52:16 +0000