[NaCl SDK] Add documentation for File I/O.

BUG=none
R=eliben@chromium.org, sbc@chromium.org

Review URL: https://codereview.chromium.org/51953011

git-svn-id: svn://svn.chromium.org/chrome/trunk/src@232213 0039d316-1c4b-4281-b951-d872f2087c98

6 files changed, 559 insertions, 14 deletions

diff --git a/native_client_sdk/src/doc/devguide/coding/FileIO.rst b/native_client_sdk/src/doc/devguide/coding/FileIO.rst
deleted file mode 100644
index 6e3a4a5..0000000
--- a/native_client_sdk/src/doc/devguide/coding/FileIO.rst
+++ /dev/null
@@ -1,11 +0,0 @@
-.. _devguide-coding-fileio:
-
-########
-File I/O
-########
-
-foo
-
-.. _quota_management:
-
-.. _enabling_file_access:
diff --git a/native_client_sdk/src/doc/devguide/coding/file-io.rst b/native_client_sdk/src/doc/devguide/coding/file-io.rst
new file mode 100644
index 0000000..fd2a38c
--- /dev/null
+++ b/native_client_sdk/src/doc/devguide/coding/file-io.rst
@@ -0,0 +1,556 @@
+.. _devguide-coding-fileio:
+
+########
+File I/O
+########
+
+.. contents::
+  :local:
+  :backlinks: none
+  :depth: 2
+
+Introduction
+============
+
+This chapter describes how to use the `FileIO API
+<https://developers.google.com/native-client/peppercpp/classpp_1_1_file_i_o>`_
+to read and write files using a local secure data store.
+
+You might use the File IO API with the URL Loading APIs to create an overall
+data download and caching solution for your NaCl applications. For example:
+
+#. Use the File IO APIs to check the local disk to see if a file exists that
+   your program needs.
+#. If the file exists locally, load it into memory using the File IO API. If
+   the file doesn't exist locally, use the URL Loading API to retrieve the
+   file from the server.
+#. Use the File IO API to write the file to disk.
+#. Load the file into memory using the File IO API when needed by your
+   application.
+
+The example discussed in this chapter is included in the SDK in the directory
+``examples/api/file_io``.
+
+Reference information
+=====================
+
+For reference information related to FileIO, see the following documentation:
+
+* `file_io.h
+  <https://developers.google.com/native-client/peppercpp/file__io_8h>`_ - API
+  to create a FileIO object
+* `file_ref.h
+  <https://developers.google.com/native-client/peppercpp/file__ref_8h>`_ - API
+  to create a file reference or "weak pointer" to a file in a file system
+* `file_system.h
+  <https://developers.google.com/native-client/peppercpp/file__system_8h>`_ -
+  API to create a file system associated with a file
+
+Local file I/O
+==============
+
+Chrome provides an obfuscated, restricted area on disk to which a web app can
+safely `read and write files
+<https://developers.google.com/chrome/whitepapers/storage#persistent>`_. The
+Pepper FileIO, FileRef, and FileSystem APIs (collectively called the File IO
+APIs) allow you to access this sandboxed local disk so you can read and write
+files and manage caching yourself. The data is persistent between launches of
+Chrome, and is not removed unless your application deletes it or the user
+manually deletes it. There is no limit to the amount of local data you can
+use, other than the actual space available on the local drive.
+
+.. _quota_management:
+.. _enabling_file_access:
+
+Enabling local file I/O
+-----------------------
+
+The easiest way to enable the writing of persistent local data is to include
+the `unlimitedStorage permission
+<http://developer.chrome.com/extensions/declare_permissions.html#unlimitedStorage>`_
+in your Chrome Web Store manifest file. With this permission you can use the
+Pepper FileIO API without the need to request disk space at run time. When
+the user installs the app Chrome displays a message announcing that the app
+writes to the local disk.
+
+If you do not use the ``unlimitedStorage`` permission you must include
+JavaScript code that calls the `HTML5 Quota Management API
+<http://updates.html5rocks.com/2011/11/Quota-Management-API-Fast-Facts>`_ to
+explicitly request local disk space before using the FileIO API. In this case
+Chrome will prompt the user to accept a requestQuota call every time one is
+made.
+
+Testing local file I/O
+----------------------
+
+You should be aware that using the ``unlimitedStorage`` manifest permission
+constrains the way you can test your app. Three of the four techniques
+described in :doc:`Running Native Client Applications <../devcycle/running>`
+read the Chrome Web Store manifest file and enable the ``unlimitedStorage``
+permission when it appears, but the first technique (local server) does not.
+If you want to test the file IO portion of your app with a simple local server,
+you need to include JavaScript code that calls the HTML5 Quota Management API.
+When you deliver your application you can replace this code with the
+``unlimitedStorage`` manifest permission.
+
+The ``file_io`` example
+=======================
+
+The Native Client SDK includes an example, ``file_io``, that demonstrates how
+to read and write a local disk file. Since you will probably run the example
+from a local server without a Chrome Web Store manifest file, the example's
+index file uses JavaScript to perform the Quota Management setup as described
+above. The example has these primary files:
+
+* ``index.html`` - The HTML code that launches the Native Client module and
+  displays the user interface.
+* ``example.js`` - JavaScript code that requests quota (as described above). It
+  also listens for user interaction with the user interface, and forwards the
+  requests to the Native Client module.
+* ``file_io.cc`` - The code that sets up and provides an entry point to the
+  Native Client module.
+
+The remainder of this section covers the code in the ``file_io.cc`` file for
+reading and writing files.
+
+File I/O overview
+-----------------
+
+Like many Pepper APIs, the File IO API includes a set of methods that execute
+asynchronously and that invoke callback functions in your Native Client module.
+Unlike most other examples, the ``file_io`` example also demonstrates how to
+make Pepper calls synchronously on a worker thread.
+
+It is illegal to make blocking calls to Pepper on the module's main thread.
+This restriction is lifted when running on a worker thread---this is called
+"calling Pepper off the main thread". This often simplifies the logic of your
+code; multiple asynchronous Pepper functions can be called from one function on
+your worker thread, so you can use the stack and standard control flow
+structures normally.
+
+The high-level flow for the ``file_io`` example is described below.  Note that
+methods in the namespace ``pp`` are part of the Pepper C++ API.
+
+Creating and writing a file
+---------------------------
+
+Following are the high-level steps involved in creating and writing to a
+file:
+
+#. ``pp::FileIO::Open`` is called with the ``PP_FILEOPEN_FLAG_CREATE`` flag to
+   create a file.  Because the callback function is ``pp::BlockUntilComplete``,
+   this thread is blocked until ``Open`` succeeds or fails.
+#. ``pp::FileIO::Write`` is called to write the contents. Again, the thread is
+   blocked until the call to ``Write`` completes. If there is more data to
+   write, ``Write`` is called again.
+#. When there is no more data to write, call ``pp::FileIO::Flush``.
+
+Opening and reading a file
+--------------------------
+
+Following are the high-level steps involved in opening and reading a file:
+
+#. ``pp::FileIO::Open`` is called to open the file. Because the callback
+   function is ``pp::BlockUntilComplete``, this thread is blocked until Open
+   succeeds or fails.
+#. ``pp::FileIO::Query`` is called to query information about the file, such as
+   its file size. The thread is blocked until ``Query`` completes.
+#. ``pp::FileIO::Read`` is called to read the contents. The thread is blocked
+   until ``Read`` completes. If there is more data to read, ``Read`` is called
+   again.
+
+Deleting a file
+---------------
+
+Deleting a file is straightforward: call ``pp::FileRef::Delete``. The thread is
+blocked until ``Delete`` completes.
+
+Making a directory
+------------------
+
+Making a directory is also straightforward: call ``pp::File::MakeDirectory``.
+The thread is blocked until ``MakeDirectory`` completes.
+
+Listing the contents of a directory
+-----------------------------------
+
+Following are the high-level steps involved in listing a directory:
+
+#. ``pp::FileRef::ReadDirectoryEntries`` is called, and given a directory entry
+   to list. A callback is given as well; many of the other functions use
+   ``pp::BlockUntilComplete``, but ``ReadDirectoryEntries`` returns results in
+   its callback, so it must be specified.
+#. When the call to ``ReadDirectoryEntries`` completes, it calls
+   ``ListCallback`` which packages up the results into a string message, and
+   sends it to JavaScript.
+
+``file_io`` deep dive
+=====================
+
+The ``file_io`` example displays a user interface with a couple of fields and
+several buttons. Following is a screenshot of the ``file_io`` example:
+
+.. image:: /images/fileioexample.png
+
+Each radio button is a file operation you can perform, with some reasonable
+default values for filenames. Try typing a message in the large input box and
+clicking ``Save``, then switching to the ``Load File`` operation, and
+clicking ``Load``.
+
+Let's take a look at what is going on under the hood.
+
+Opening a file system and preparing for file I/O
+------------------------------------------------
+
+``pp::Instance::Init`` is called when an instance of a module is created. In
+this example, ``Init`` starts a new thread (via the ``pp::SimpleThread``
+class), and tells it to open the filesystem:
+
+.. naclcode::
+
+  virtual bool Init(uint32_t /*argc*/,
+                    const char * /*argn*/ [],
+                    const char * /*argv*/ []) {
+    file_thread_.Start();
+    // Open the file system on the file_thread_. Since this is the first
+    // operation we perform there, and because we do everything on the
+    // file_thread_ synchronously, this ensures that the FileSystem is open
+    // before any FileIO operations execute.
+    file_thread_.message_loop().PostWork(
+        callback_factory_.NewCallback(&FileIoInstance::OpenFileSystem));
+    return true;
+  }
+
+When the file thread starts running, it will call ``OpenFileSystem``. This
+calls ``pp::FileSystem::Open`` and blocks the file thread until the function
+returns.
+
+.. Note::
+  :class: note
+
+  Note that the call to ``pp::FileSystem::Open`` uses
+  ``pp::BlockUntilComplete`` as its callback. This is only possible because we
+  are running off the main thread; if you try to make a blocking call from the
+  main thread, the function will return the error
+  ``PP_ERROR_BLOCKS_MAIN_THREAD``.
+
+.. naclcode::
+
+  void OpenFileSystem(int32_t /*result*/) {
+    int32_t rv = file_system_.Open(1024 * 1024, pp::BlockUntilComplete());
+    if (rv == PP_OK) {
+      file_system_ready_ = true;
+      // Notify the user interface that we're ready
+      PostMessage("READY|");
+    } else {
+      ShowErrorMessage("Failed to open file system", rv);
+    }
+  }
+
+Handling messages from JavaScript
+---------------------------------
+
+When you click the ``Save`` button, JavaScript posts a message to the NaCl
+module with the file operation to perform sent as a string (See :doc:`Messaging
+System <message-system>` for more details on message passing). The string is
+parsed by ``HandleMessage``, and new work is added to the file thread:
+
+.. naclcode::
+
+  virtual void HandleMessage(const pp::Var& var_message) {
+    if (!var_message.is_string())
+      return;
+
+    // Parse message into: instruction file_name_length file_name [file_text]
+    std::string message = var_message.AsString();
+    std::string instruction;
+    std::string file_name;
+    std::stringstream reader(message);
+    int file_name_length;
+
+    reader >> instruction >> file_name_length;
+    file_name.resize(file_name_length);
+    reader.ignore(1);  // Eat the delimiter
+    reader.read(&file_name[0], file_name_length);
+
+    ...
+
+    // Dispatch the instruction
+    if (instruction == kLoadPrefix) {
+      file_thread_.message_loop().PostWork(
+          callback_factory_.NewCallback(&FileIoInstance::Load, file_name));
+    } else if (instruction == kSavePrefix) {
+      ...
+    }
+  }
+
+Saving a file
+-------------
+
+``FileIoInstance::Save`` is called when the ``Save`` button is pressed. First,
+it checks to see that the FileSystem has been successfully opened:
+
+.. naclcode::
+
+  if (!file_system_ready_) {
+    ShowErrorMessage("File system is not open", PP_ERROR_FAILED);
+    return;
+  }
+
+It then creates a ``pp::FileRef`` resource with the name of the file. A
+``FileRef`` resource is a weak reference to a file in the FileSystem; that is,
+a file can still be deleted even if there are outstanding ``FileRef``
+resources.
+
+.. naclcode::
+
+  pp::FileRef ref(file_system_, file_name.c_str());
+
+Next, a ``pp::FileIO`` resource is created and opened. The call to
+``pp::FileIO::Open`` passes ``PP_FILEOPEFLAG_WRITE`` to open the file for
+writing, ``PP_FILEOPENFLAG_CREATE`` to create a new file if it doesn't already
+exist and ``PP_FILEOPENFLAG_TRUNCATE`` to clear the file of any previous
+content:
+
+.. naclcode::
+
+  pp::FileIO file(this);
+
+  int32_t open_result =
+      file.Open(ref,
+                PP_FILEOPENFLAG_WRITE | PP_FILEOPENFLAG_CREATE |
+                    PP_FILEOPENFLAG_TRUNCATE,
+                pp::BlockUntilComplete());
+  if (open_result != PP_OK) {
+    ShowErrorMessage("File open for write failed", open_result);
+    return;
+  }
+
+Now that the file is opened, it is written to in chunks. In an asynchronous
+model, this would require writing a separate function, storing the current
+state on the free store and a chain of callbacks. Because this function is
+called off the main thread, ``pp::FileIO::Write`` can be called synchronously
+and a conventional do/while loop can be used:
+
+.. naclcode::
+
+  int64_t offset = 0;
+  int32_t bytes_written = 0;
+  do {
+    bytes_written = file.Write(offset,
+                               file_contents.data() + offset,
+                               file_contents.length(),
+                               pp::BlockUntilComplete());
+    if (bytes_written > 0) {
+      offset += bytes_written;
+    } else {
+      ShowErrorMessage("File write failed", bytes_written);
+      return;
+    }
+  } while (bytes_written < static_cast<int64_t>(file_contents.length()));
+
+Finally, the file is flushed to push all changes to disk:
+
+.. naclcode::
+
+  int32_t flush_result = file.Flush(pp::BlockUntilComplete());
+  if (flush_result != PP_OK) {
+    ShowErrorMessage("File fail to flush", flush_result);
+    return;
+  }
+
+Loading a file
+--------------
+
+``FileIoInstance::Load`` is called when the ``Load`` button is pressed. Like
+the ``Save`` function, ``Load`` first checks to see if the FileSystem has been
+successfully opened, and creates a new ``FileRef``:
+
+.. naclcode::
+
+  if (!file_system_ready_) {
+    ShowErrorMessage("File system is not open", PP_ERROR_FAILED);
+    return;
+  }
+  pp::FileRef ref(file_system_, file_name.c_str());
+
+Next, ``Load`` creates and opens a new ``FileIO`` resource, passing
+``PP_FILEOPENFLAG_READ`` to open the file for reading. The result is compared
+to ``PP_ERROR_FILENOTFOUND`` to give a better error message when the file
+doesn't exist:
+
+.. naclcode::
+
+  int32_t open_result =
+      file.Open(ref, PP_FILEOPENFLAG_READ, pp::BlockUntilComplete());
+  if (open_result == PP_ERROR_FILENOTFOUND) {
+    ShowErrorMessage("File not found", open_result);
+    return;
+  } else if (open_result != PP_OK) {
+    ShowErrorMessage("File open for read failed", open_result);
+    return;
+  }
+
+Then ``Load`` calls ``pp::FileIO::Query`` to get metadata about the file, such
+as its size. This is used to allocate a ``std::vector`` buffer that holds the
+data from the file in memory:
+
+.. naclcode::
+
+  int32_t query_result = file.Query(&info, pp::BlockUntilComplete());
+  if (query_result != PP_OK) {
+    ShowErrorMessage("File query failed", query_result);
+    return;
+  }
+
+  ...
+
+  std::vector<char> data(info.size);
+
+Similar to ``Save``, a conventional while loop is used to read the file into
+the newly allocated buffer:
+
+.. naclcode::
+
+  int64_t offset = 0;
+  int32_t bytes_read = 0;
+  int32_t bytes_to_read = info.size;
+  while (bytes_to_read > 0) {
+    bytes_read = file.Read(offset,
+                           &data[offset],
+                           data.size() - offset,
+                           pp::BlockUntilComplete());
+    if (bytes_read > 0) {
+      offset += bytes_read;
+      bytes_to_read -= bytes_read;
+    } else if (bytes_read < 0) {
+      // If bytes_read < PP_OK then it indicates the error code.
+      ShowErrorMessage("File read failed", bytes_read);
+      return;
+    }
+  }
+
+Finally, the contents of the file are sent back to JavaScript, to be displayed
+on the page. This example uses "``DISP|``" as a prefix command for display
+information:
+
+.. naclcode::
+
+  std::string string_data(data.begin(), data.end());
+  PostMessage("DISP|" + string_data);
+  ShowStatusMessage("Load success");
+
+Deleting a file
+---------------
+
+``FileIoInstance::Delete`` is called when the ``Delete`` button is pressed.
+First, it checks whether the FileSystem has been opened, and creates a new
+``FileRef``:
+
+.. naclcode::
+
+  if (!file_system_ready_) {
+    ShowErrorMessage("File system is not open", PP_ERROR_FAILED);
+    return;
+  }
+  pp::FileRef ref(file_system_, file_name.c_str());
+
+Unlike ``Save`` and ``Load``, ``Delete`` is called on the ``FileRef`` resource,
+not a ``FileIO`` resource. Note that the result is checked for
+``PP_ERROR_FILENOTFOUND`` to give a better error message when trying to delete
+a non-existent file:
+
+.. naclcode::
+
+  int32_t result = ref.Delete(pp::BlockUntilComplete());
+  if (result == PP_ERROR_FILENOTFOUND) {
+    ShowStatusMessage("File/Directory not found");
+    return;
+  } else if (result != PP_OK) {
+    ShowErrorMessage("Deletion failed", result);
+    return;
+  }
+
+Listing files in a directory
+----------------------------
+
+``FileIoInstance::List`` is called when the ``List Directory`` button is
+pressed. Like all other operations, it checks whether the FileSystem has been
+opened and creates a new ``FileRef``:
+
+.. naclcode::
+
+  if (!file_system_ready_) {
+    ShowErrorMessage("File system is not open", PP_ERROR_FAILED);
+    return;
+  }
+
+  pp::FileRef ref(file_system_, dir_name.c_str());
+
+Unlike the other operations, it does not make a blocking call to
+``pp::FileRef::ReadDirectoryEntries``. Since ``ReadDirectoryEntries`` returns
+the resulting directory entries in its callback, a new callback object is
+created pointing to ``FileIoInstance::ListCallback``.
+
+The ``pp::CompletionCallbackFactory`` template class is used to instantiate a
+new callback. Notice that the ``FileRef`` resource is passed as a parameter;
+this will add a reference count to the callback object, to keep the ``FileRef``
+resource from being destroyed when the function finishes.
+
+.. naclcode::
+
+  // Pass ref along to keep it alive.
+  ref.ReadDirectoryEntries(callback_factory_.NewCallbackWithOutput(
+      &FileIoInstance::ListCallback, ref));
+
+``FileIoInstance::ListCallback`` then gets the results passed as a
+``std::vector`` of ``pp::DirectoryEntry`` objects, and sends them to
+JavaScript:
+
+.. naclcode::
+
+  void ListCallback(int32_t result,
+                    const std::vector<pp::DirectoryEntry>& entries,
+                    pp::FileRef /*unused_ref*/) {
+    if (result != PP_OK) {
+      ShowErrorMessage("List failed", result);
+      return;
+    }
+
+    std::stringstream ss;
+    ss << "LIST";
+    for (size_t i = 0; i < entries.size(); ++i) {
+      pp::Var name = entries[i].file_ref().GetName();
+      if (name.is_string()) {
+        ss << "|" << name.AsString();
+      }
+    }
+    PostMessage(ss.str());
+    ShowStatusMessage("List success");
+  }
+
+Making a new directory
+----------------------
+
+``FileIoInstance::MakeDir`` is called when the ``Make Directory`` button is
+pressed. Like all other operations, it checks whether the FileSystem has been
+opened and creates a new ``FileRef``:
+
+.. naclcode::
+
+  if (!file_system_ready_) {
+    ShowErrorMessage("File system is not open", PP_ERROR_FAILED);
+    return;
+  }
+  pp::FileRef ref(file_system_, dir_name.c_str());
+
+Then the ``pp::FileRef::MakeDirectory`` function is called.
+
+.. naclcode::
+
+  int32_t result = ref.MakeDirectory(pp::BlockUntilComplete());
+  if (result != PP_OK) {
+    ShowErrorMessage("Make directory failed", result);
+    return;
+  }
+  ShowStatusMessage("Make directory success");
diff --git a/native_client_sdk/src/doc/devguide/coding/nacl_io.rst b/native_client_sdk/src/doc/devguide/coding/nacl_io.rst
index 6cee0f7..d07eb66 100644
--- a/native_client_sdk/src/doc/devguide/coding/nacl_io.rst
+++ b/native_client_sdk/src/doc/devguide/coding/nacl_io.rst
@@ -23,7 +23,7 @@ can be used by the application. For example, the Chrome browser supports the
 <http://www.html5rocks.com/en/tutorials/file/filesystem/>`_ which provides
 access to a protected area of the local file system. This filesystem can
 be accessed by an HTML page using JavaScript commands, and also by a Native
-Client module using the Pepper :doc:`File IO API <FileIO>`. With nacl_io
+Client module using the Pepper :doc:`File IO API <file-io>`. With nacl_io
 a Native Client application can mount an HTML5 filesystem and access it via
 standard POSIX I/O function such as ``fopen``, ``fseek``, ``fread``,
 ``fwrite``, and ``fclose``, or their low level UNIX counterparts ``open``,
diff --git a/native_client_sdk/src/doc/images/fileioexample.png b/native_client_sdk/src/doc/images/fileioexample.png
index b6ba42c..114ccb8 100644
--- a/native_client_sdk/src/doc/images/fileioexample.png
+++ b/native_client_sdk/src/doc/images/fileioexample.png
diff --git a/native_client_sdk/src/doc/index.rst b/native_client_sdk/src/doc/index.rst
index d3da200..2b32153 100644
--- a/native_client_sdk/src/doc/index.rst
+++ b/native_client_sdk/src/doc/index.rst
@@ -29,7 +29,7 @@ Contents:
    devguide/coding/audio.rst
    devguide/coding/application-structure.rst
    devguide/coding/native-client-modules.rst
-   devguide/coding/FileIO.rst
+   devguide/coding/file-io.rst
    devguide/coding/nacl_io.rst
    devguide/coding/message-system.rst
    devguide/coding/progress-events.rst
diff --git a/native_client_sdk/src/doc/overview.rst b/native_client_sdk/src/doc/overview.rst
index e8ad2d7..d1c6e89 100644
--- a/native_client_sdk/src/doc/overview.rst
+++ b/native_client_sdk/src/doc/overview.rst
@@ -243,7 +243,7 @@ capabilities, including:
 
 * :doc:`Talking to the JavaScript code in your application
   <devguide/coding/message-system>` from the C++ code in your NaCl module.
-* :doc:`Doing file I/O <devguide/coding/FileIO>`.
+* :doc:`Doing file I/O <devguide/coding/file-io>`.
 * :doc:`Playing audio <devguide/coding/audio>`.
 * :doc:`Rendering 3D graphics <devguide/coding/3D-graphics>`.