Merge "Update kernel header generation docs."

2 files changed, 107 insertions, 148 deletions

diff --git a/libc/kernel/README.TXT b/libc/kernel/README.TXT
index e4c11a1..0edbcc6 100644
--- a/libc/kernel/README.TXT
+++ b/libc/kernel/README.TXT
@@ -1,93 +1,47 @@
-Bionic comes with a set of 'clean' Linux kernel headers that can safely be
-included by userland applications and libraries without fear of hideous
-conflicts. for more information why this is needed, see the "RATIONALE"
-section at the end of this document.
-
-these clean headers are automatically generated by several scripts located
-in the 'bionic/kernel/tools' directory, which process a set of original
-and unmodified kernel headers in order to get rid of many annoying
+Bionic comes with a processed set of all of the uapi Linux kernel headers that
+can safely be included by userland applications and libraries.
+
+These clean headers are automatically generated by several scripts located
+in the 'bionic/kernel/tools' directory. The tools process the original
+unmodified kernel headers in order to get rid of many annoying
 declarations and constructs that usually result in compilation failure.
 
-the 'clean headers' only contain type and macro definitions, with the
+The 'clean headers' only contain type and macro definitions, with the
 exception of a couple static inline functions used for performance
-reason (e.g. optimized CPU-specific byte-swapping routines)
+reason (e.g. optimized CPU-specific byte-swapping routines).
 
-they can be included from C++, or when compiling code in strict ANSI mode.
-they can be also included before or after any Bionic C library header.
+They can be included from C++, or when compiling code in strict ANSI mode.
+They can be also included before or after any Bionic C library header.
 
-the generation process works as follows:
+Description of the directories involved in generating the parsed kernel headers:
 
   * 'external/kernel-headers/original/'
-    contains a set of kernel headers as normally found in the 'include'
-    directory of a normal Linux kernel source tree. note that this should
-    only contain the files that are really needed by Android (use
-    'find_headers.py' to find these automatically).
-
-  * 'bionic/libc/kernel/common'
-    contains the non-arch-specific clean headers and directories
-    (e.g. linux, asm-generic and mtd)
-
-  * 'bionic/libc/kernel/arch-arm/'
-    contains the ARM-specific directory tree of clean headers.
-
-  * 'bionic/libc/kernel/arch-arm/asm'
-    contains the real ARM-specific headers
-
-  * 'bionic/libc/kernel/arch-x86'
-    'bionic/libc/kernel/arch-x86/asm'
-    similarly contains all headers and symlinks to be used on x86
+    Contains the uapi kernel headers found in the android kernel. Note this
+    also includes the header files that are generated by building the kernel
+    sources.
 
-  * 'bionic/libc/kernel/tools' contains various Python and shell scripts used
-    to manage and re-generate the headers
+  * 'bionic/libc/kernel/uapi'
+    Contains the cleaned kernel headers and mirrors the directory structure
+    in 'external/kernel-headers/original/uapi/'.
 
-the tools you can use are:
+  * 'bionic/libc/kernel/tools'
+    Contains various Python and shell scripts used to get and re-generate
+    the headers.
 
-  * tools/find_users.py
-    scans a list of source files or directories and prints which ones do
-    include Linux headers.
+The tools to get/parse the headers:
 
-  * tools/find_headers.py
-    scans a list of source files or directories and recursively finds all
-    the original kernel headers they need.
+  * tools/generate_uapi_headers.sh
+    Checks out the android kernel and generates all uapi header files.
+    copies all the changed files into external/kernel-headers.
 
   * tools/clean_header.py
-    prints the clean version of a given kernel header. with the -u option,
+    Prints the clean version of a given kernel header. With the -u option,
     this will also update the corresponding clean header file if its
-    content has changed. you can also process more than one file with -u
+    content has changed. You can also process more than one file with -u.
 
   * tools/update_all.py
-    automatically update all clean headers from the content of 
-    'external/kernel-headers/original'. this is the script you're likely going to
-    run whenever you update the original headers.
-
-
-HOW TO BUILD BIONIC AND OTHER PROGRAMS WITH THE CLEAN HEADERS:
-==============================================================
-
-add bionic/kernel/common and bionic/kernel/arch-<yourarch> to your C
-include path. that should be enough. Note that Bionic will not compile properly 
-if you don't.
-
-
-HOW TO SUPPORT ANOTHER ARCHITECTURE:
-====================================
-
-see the content of tools/defaults.py, you will need to make a few updates
-here:
-
-  - add a new item to the 'kernel_archs' list of supported architectures
-
-  - add a proper definition for 'kernel_known_<arch>_statics' with
-    relevant definitions.
-
-  - update 'kernel_known_statics' to map "<arch>" to
-    'kernel_known_<arch>_statics'
-
-then, add the new architecture-specific headers to original/asm-<arch>.
-(please ensure that these are really needed, e.g. with tools/find_headers.py)
-
-finally, run tools/update_all.py
-
+    Automatically update all clean headers from the content of
+    'external/kernel-headers/original'.
 
 
 HOW TO UPDATE THE HEADERS WHEN NEEDED:
@@ -99,81 +53,13 @@ IMPORTANT IMPORTANT:
   NOT BREAK THE KERNEL <-> USER ABI, FOR EXAMPLE BY CHANGING THE SIZE
   OF A GIVEN TYPE. THIS TASK CANNOT BE EASILY AUTOMATED AT THE MOMENT
 
-copy any updated kernel header into the corresponding location under
-'bionic/kernel/original'.
-
-for any new kernel header you want to add, first run tools/find_headers.py to be
-sure that it is really needed by the Android sources. then add it to
-'bionic/kernel/original'
-
-then, run tools/update_all.py to re-run the auto-cleaning
-
-
-
-HOW THE CLEANUP PROCESS WORKS:
-==============================
-
-this section describes the action performed by the cleanup program(s) when they
-process the original kernel headers into clean ones:
-
-1. Optimize well-known macros (e.g. __KERNEL__, __KERNEL_STRICT_NAMES)
-
-    this pass gets rid of everything that is guarded by a well-known macro
-    definition. this means that a block like
-
-       #ifdef __KERNEL__
-       ....
-       #endif
-
-    will be totally omitted from the output. the optimizer is smart enough to
-    handle all complex C-preprocessor conditional expression appropriately.
-    this means that, for example:
-
-       #if defined(__KERNEL__) || defined(FOO)
-       ...
-       #endif
-
-    will be transformed into:
-
-       #ifdef FOO
-       ...
-       #endif
-
-    see tools/defaults.py for the list of well-known macros used in this pass,
-    in case you need to update it in the future.
-
-    note that this also remove any reference to a kernel-specific configuration
-    macro like CONFIG_FOO from the clean headers.
-
-
-2. remove variable and function declarations:
-
-  this pass scans non-directive text and only keeps things that look like a
-  typedef/struct/union/enum declaration. this allows to get rid of any variable
-  or function declaration that should only be used within the kernel anyway
-  (and which normally *should* be guarded in a #ifdef __KERNEL__ ... #endif
-  block, if the kernel writers were not so messy)
-
-  there are however a few exceptions: it is seldom useful to keep the definition
-  of some static inline functions performing very simple operations. a good
-  example is the optimized 32-bit byte-swap function found in
-  arch-arm/asm/byteorder.h
-
-  the list of exceptions is in tools/defaults.py in case you need to update it
-  in the future.
-
-  note that we do *not* remove macro definitions, including these macro that
-  perform a call to one of these kernel-header functions, or even define other
-  functions. we consider it safe since userland applications have no business
-  using them anyway.
-
-
-3. whitespace cleanup:
+Grab the latest headers from the android kernel by running this command:
 
-  the final pass remove any comments and empty lines from the final headers.
+  bionic/kernel/tools/generate_uapi_headers.sh --download-kernel
 
+Next, run this command to copy the parsed files to bionic/libc/kernel/uapi:
 
-4. add a standard disclaimer:
+  bionic/kernel/tools/update_all.py
 
-  prepended to each generated header, contains a message like
-  "do not edit directly - file was auto-generated by ...."
+After this, you will need to build/test the tree to make sure that these
+changes do not introduce any errors.
diff --git a/libc/kernel/tools/clean_header.py b/libc/kernel/tools/clean_header.py
index 238087b..6601817 100755
--- a/libc/kernel/tools/clean_header.py
+++ b/libc/kernel/tools/clean_header.py
@@ -1,5 +1,78 @@
 #!/usr/bin/env python
+
+#------------------------------------------------------------------------------
+# Description of the header clean process
+#------------------------------------------------------------------------------
+# Here is the list of actions performed by this script to clean the original
+# kernel headers.
+#
+# 1. Optimize well-known macros (e.g. __KERNEL__, __KERNEL_STRICT_NAMES)
+#
+#     This pass gets rid of everything that is guarded by a well-known macro
+#     definition. This means that a block like:
+#
+#        #ifdef __KERNEL__
+#        ....
+#        #endif
+#
+#     Will be totally omitted from the output. The optimizer is smart enough to
+#     handle all complex C-preprocessor conditional expression appropriately.
+#     This means that, for example:
+#
+#        #if defined(__KERNEL__) || defined(FOO)
+#        ...
+#        #endif
+#
+#     Will be transformed into:
+#
+#        #ifdef FOO
+#        ...
+#        #endif
+#
+#     See tools/defaults.py for the list of well-known macros used in this pass,
+#     in case you need to update it in the future.
+#
+#     Note that this also removes any reference to a kernel-specific
+#     configuration macro like CONFIG_FOO from the clean headers.
+#
+#
+# 2. Remove variable and function declarations:
+#
+#   This pass scans non-directive text and only keeps things that look like a
+#   typedef/struct/union/enum declaration. This allows us to get rid of any
+#   variables or function declarations that should only be used within the
+#   kernel anyway (and which normally *should* be guarded by an #ifdef
+#   __KERNEL__ ...  #endif block, if the kernel writers were not so messy).
+#
+#   There are, however, a few exceptions: it is seldom useful to keep the
+#   definition of some static inline functions performing very simple
+#   operations. A good example is the optimized 32-bit byte-swap function
+#   found in:
+#
+#     arch-arm/asm/byteorder.h
+#
+#   The list of exceptions is in tools/defaults.py in case you need to update
+#   it in the future.
+#
+#   Note that we do *not* remove macro definitions, including these macro that
+#   perform a call to one of these kernel-header functions, or even define other
+#   functions. We consider it safe since userland applications have no business
+#   using them anyway.
+#
+#
+# 3. Whitespace cleanup:
+#
+#   The final pass removes any comments and empty lines from the final headers.
+#
+#
+# 4. Add a standard disclaimer:
+#
+#   The message:
+#
+#   /* WARNING: DO NOT EDIT, AUTO-GENERATED CODE - SEE TOP FOR INSTRUCTIONS */
 #
+#   Is prepended to each generated header.
+#------------------------------------------------------------------------------
 
 import sys, cpp, kernel, glob, os, re, getopt
 from defaults import *