Allow comments in extension config files.

Added a script to remove comments from the extension api JSON files before
processing for the extension docs. We will now be able to comment in the JSON
files, and they should be much easier to read. Also added the license header
to all the JSON files.

BUG=114233
TEST=Put comments in one of the JSON files and remake the docs. They will make
with no problems.


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=124660

Review URL: http://codereview.chromium.org/9447090

git-svn-id: svn://svn.chromium.org/chrome/trunk/src@124878 0039d316-1c4b-4281-b951-d872f2087c98

5 files changed, 261 insertions, 0 deletions

diff --git a/third_party/json_minify/README b/third_party/json_minify/README
new file mode 100644
index 0000000..98406f2
--- /dev/null
+++ b/third_party/json_minify/README
@@ -0,0 +1,35 @@
+/*! JSON.minify()
+	v0.1 (c) Kyle Simpson
+	MIT License
+*/
+
+JSON.minify() minifies blocks of JSON-like content into valid JSON by removing all 
+whitespace *and* comments.
+
+JSON parsers (like JavaScript's JSON.parse() parser) generally don't consider JSON
+with comments to be valid and parseable. So, the intended usage is to minify 
+development-friendly JSON (with comments) to valid JSON before parsing, such as:
+
+JSON.parse(JSON.minify(str));
+
+Now you can maintain development-friendly JSON documents, but minify them before
+parsing or before transmitting them over-the-wire.
+
+Though comments are not officially part of the JSON standard, this post from
+Douglas Crockford back in late 2005 helps explain the motivation behind this project.
+
+http://tech.groups.yahoo.com/group/json/message/152
+
+"A JSON encoder MUST NOT output comments. A JSON decoder MAY accept and ignore comments."
+
+Basically, comments are not in the JSON *generation* standard, but that doesn't mean
+that a parser can't be taught to ignore them. Which is exactly what JSON.minify()
+is for.
+
+The first implementation of JSON.minify() is in JavaScript, but the intent is to
+port the implementation to as many other environments as possible/practical.
+
+NOTE: As transmitting bloated (ie, with comments/whitespace) JSON would be wasteful
+and silly, this JSON.minify() is intended for use in server-side processing
+environments where you can strip comments/whitespace from JSON before parsing
+a JSON document, or before transmitting such over-the-wire from server to browser.
\ No newline at end of file
diff --git a/third_party/json_minify/README.chromium b/third_party/json_minify/README.chromium
new file mode 100644
index 0000000..0f5f630
--- /dev/null
+++ b/third_party/json_minify/README.chromium
@@ -0,0 +1,17 @@
+Name: JSON.minify
+URL: https://github.com/getify/JSON.minify
+and also https://github.com/kitcambridge/JSON.minify
+Version: 0.1
+License: MIT License
+License file: https://github.com/getify/JSON.minify/blob/master/README.txt
+(README has license info)
+Security Critical: no
+
+Description:
+A set of scripts that remove comments and whitespace from JSON files.
+
+Local Modifications:
+- Added the __init__.py file for easier imports
+- Got the (much faster) json-minify-sans-regexp.js file from the second
+  URL listed
+- Small change to minify_json.py to pass license tests
diff --git a/third_party/json_minify/__init__.py b/third_party/json_minify/__init__.py
new file mode 100644
index 0000000..45bce20
--- /dev/null
+++ b/third_party/json_minify/__init__.py
@@ -0,0 +1,5 @@
+# Copyright (c) 2012 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from minify_json import json_minify
diff --git a/third_party/json_minify/minify-sans-regexp.js b/third_party/json_minify/minify-sans-regexp.js
new file mode 100644
index 0000000..99c817c
--- /dev/null
+++ b/third_party/json_minify/minify-sans-regexp.js
@@ -0,0 +1,91 @@
+/*!
+ * `JSON.minify()`
+ * This version does not use regular expressions.
+ *
+ * Copyright 2011, Kyle Simpson.
+ * Copyright 2012, Kit Cambridge.
+ *
+ * Released under the MIT License.
+*/
+
+;(function () {
+  var JSON = this.JSON;
+
+  // Create the global JSON object if it doesn't exist.
+  if (Object(JSON) !== JSON) {
+    JSON = this.JSON = {};
+  }
+
+  JSON.minify = function (source) {
+    var index = 0, length = source.length, result = "", symbol, position;
+    while (index < length) {
+      symbol = source.charAt(index);
+      switch (symbol) {
+        // Ignore whitespace tokens. According to ES 5.1 section 15.12.1.1,
+        // whitespace tokens include tabs, carriage returns, line feeds, and
+        // space characters.
+        // -----------------------------------------------------------------
+        case "\t":
+        case "\r":
+        case "\n":
+        case " ":
+          index += 1;
+          break;
+        // Ignore line and block comments.
+        // -------------------------------
+        case "/":
+          symbol = source.charAt(index += 1);
+          switch (symbol) {
+            // Line comments.
+            // -------------
+            case "/":
+              position = source.indexOf("\n", index);
+              if (position < 0) {
+                // Check for CR-style line endings.
+                position = source.indexOf("\r", index);
+              }
+              index = position > -1 ? position : length;
+              break;
+            // Block comments.
+            // ---------------
+            case "*":
+              position = source.indexOf("*/", index);
+              if (position > -1) {
+                // Advance the scanner's position past the end of the comment.
+                index = position += 2;
+                break;
+              }
+              throw SyntaxError("Unterminated block comment.");
+            default:
+              throw SyntaxError("Invalid comment.");
+          }
+          break;
+        // Parse strings separately to ensure that any whitespace characters and
+        // JavaScript-style comments within them are preserved.
+        // ---------------------------------------------------------------------
+        case '"':
+          position = index;
+          while (index < length) {
+            symbol = source.charAt(index += 1);
+            if (symbol == "\\") {
+              // Skip past escaped characters.
+              index += 1;
+            } else if (symbol == '"') {
+              break;
+            }
+          }
+          if (source.charAt(index) == '"') {
+            result += source.slice(position, index += 1);
+            break;
+          }
+          throw SyntaxError("Unterminated string.");
+        // Preserve all other characters.
+        // ------------------------------
+        default:
+          result += symbol;
+          index += 1;
+      }
+    }
+    return result;
+  };
+}).call(this);
diff --git a/third_party/json_minify/minify_json.py b/third_party/json_minify/minify_json.py
new file mode 100644
index 0000000..69aac6e
--- /dev/null
+++ b/third_party/json_minify/minify_json.py
@@ -0,0 +1,113 @@
+# Use of this source code is governed by a BSD-style license (MIT)
+'''
+Created on 20/01/2011
+
+v0.1 (C) Gerald Storer
+MIT License
+
+Based on JSON.minify.js: 
+https://github.com/getify/JSON.minify
+'''
+
+import re
+
+def json_minify(json,strip_space=True):
+    tokenizer=re.compile('"|(/\*)|(\*/)|(//)|\n|\r')
+    in_string = False
+    in_multiline_comment = False
+    in_singleline_comment = False
+    
+    new_str = []
+    from_index = 0 # from is a keyword in Python
+    
+    for match in re.finditer(tokenizer,json):
+        
+        if not in_multiline_comment and not in_singleline_comment:
+            tmp2 = json[from_index:match.start()]
+            if not in_string and strip_space:
+                tmp2 = re.sub('[ \t\n\r]*','',tmp2) # replace only white space defined in standard
+            new_str.append(tmp2)
+            
+        from_index = match.end()
+        
+        if match.group() == '"' and not in_multiline_comment and not in_singleline_comment:
+            escaped = re.search('(\\\\)*$',json[:match.start()])
+            if not in_string or escaped is None or len(escaped.group()) % 2 == 0:
+                # start of string with ", or unescaped " character found to end string
+                in_string = not in_string
+            from_index -= 1 # include " character in next catch
+            
+        elif match.group() == '/*' and not in_string and not in_multiline_comment and not in_singleline_comment:
+            in_multiline_comment = True
+        elif match.group() == '*/' and not in_string and in_multiline_comment and not in_singleline_comment:
+            in_multiline_comment = False
+        elif match.group() == '//' and not in_string and not in_multiline_comment and not in_singleline_comment:
+            in_singleline_comment = True
+        elif (match.group() == '\n' or match.group() == '\r') and not in_string and not in_multiline_comment and in_singleline_comment:
+            in_singleline_comment = False
+        elif not in_multiline_comment and not in_singleline_comment and (  
+             match.group() not in ['\n','\r',' ','\t'] or not strip_space):
+                new_str.append(match.group()) 
+    
+    new_str.append(json[from_index:])
+    return ''.join(new_str)
+
+if __name__ == '__main__':
+    import json # requires Python 2.6+ to run tests
+    
+    def test_json(s):
+        return json.loads(json_minify(s))
+    
+    test1 = '''// this is a JSON file with comments
+{
+    "foo": "bar",    // this is cool
+    "bar": [
+        "baz", "bum", "zam"
+    ],
+/* the rest of this document is just fluff
+   in case you are interested. */
+    "something": 10,
+    "else": 20
+}
+
+/* NOTE: You can easily strip the whitespace and comments 
+   from such a file with the JSON.minify() project hosted 
+   here on github at http://github.com/getify/JSON.minify 
+*/
+'''
+
+    test1_res = '''{"foo":"bar","bar":["baz","bum","zam"],"something":10,"else":20}'''
+    
+    test2 = '''
+{"/*":"*/","//":"",/*"//"*/"/*/"://
+"//"}
+
+'''
+    test2_res = '''{"/*":"*/","//":"","/*/":"//"}'''
+    
+    test3 = r'''/*
+this is a 
+multi line comment */{
+
+"foo"
+:
+    "bar/*"// something
+    ,    "b\"az":/*
+something else */"blah"
+
+}
+'''
+    test3_res = r'''{"foo":"bar/*","b\"az":"blah"}'''
+    
+    test4 = r'''{"foo": "ba\"r//", "bar\\": "b\\\"a/*z", 
+    "baz\\\\": /* yay */ "fo\\\\\"*/o" 
+}
+'''
+    test4_res = r'''{"foo":"ba\"r//","bar\\":"b\\\"a/*z","baz\\\\":"fo\\\\\"*/o"}'''
+    
+    assert test_json(test1) == json.loads(test1_res),'Failed test 1'
+    assert test_json(test2) == json.loads(test2_res),'Failed test 2'
+    assert test_json(test3) == json.loads(test3_res),'Failed test 3'
+    assert test_json(test4) == json.loads(test4_res),'Failed test 4'
+    if __debug__: # Don't print passed message if the asserts didn't run
+        print 'Passed all tests'