draft: post about U-Boot on Galaxy S3

updated post about newer kernel

2 files changed, 48 insertions, 11 deletions

diff --git a/content/newer-kernel-galaxyS3.rst b/content/newer-kernel-galaxyS3.rst
index 3829fa0..435b998 100644
--- a/content/newer-kernel-galaxyS3.rst
+++ b/content/newer-kernel-galaxyS3.rst
@@ -1,36 +1,36 @@
-:title: Problems with getting a newer kernel working on the Galaxy S3 and why merging is bad 
-:date: 2016-11-18 19:54
-:tags: kernel, free software, sustainability
+:title: Problems getting a current kernel working on the Galaxy S3 and why merging is bad
+:date: 2016-11-27 19:54
+:tags: kernel, free software, sustainability, phone
 :category: General
 :author: Wolfgang Wiedmeyer
 :status: draft
 :summary: Some time ago, I started to wonder if it would be possible to update the old Linux kernel for the GT-I9300 variant (international version) of the Galaxy S3 to a more recent version.
 
-Some time ago, I started to wonder if it would be possible to update the old Linux kernel for the GT-I9300 variant (international version) of the Galaxy S3 to a more recent version. A newer kernel could not only bring new features and speed improvements, but also lots of security fixes that are not yet patched. Furthermore, porting a more recent Android version to the device gets much more easier. Getting a newer kernel working with an older user space is much easier than making an old kernel work with a new user space. And isn't being able to use the newest and shiny software on arbitrary hardware, regardless of its age, the spirit of free software?
+Some time ago, I started to wonder wheter it would be possible to update the old Linux kernel for the GT-I9300 variant (international version) of the Galaxy S3 to a more recent version. A newer kernel could not only bring new features and speed improvements, but also lots of security fixes that are not yet patched. Furthermore, porting a more recent Android version to the device would be much more easier. Making a newer kernel work with an older user space is much easier than making an old kernel work with a new user space. And isn't it the spirit of free software to be able to use the newest and shiny software on arbitrary hardware, regardless of its age?
 
-But why the Galaxy S3? Wouldn't it be better to take a more recent device that already was released with a newer kernel? I work on the Galaxy S3 because I use it as my daily driver and I already have a spare device that I can use for development. Besides the Galaxy S2, it is still the best supported device in `Replicant <http://www.replicant.us/>`_ which was my original reason to start working with this phone. And especially in regards to the mainline kernel, it actually turns out to be a really good choice, but more about that in a `second post <{filename}/mainline-kernel-galaxyS3.rst>`_.
+But why Galaxy S3? Wouldn't it be better to take a more recent device that was already released with a newer kernel? I work on the Galaxy S3 because I use it as my daily driver and I have a spare device that I can use for development. Besides the Galaxy S2, it is still the best supported device by `Replicant <http://www.replicant.us/>`_ which was my original reason to start working with this phone. And especially in regards to the mainline kernel, it actually turns out to be a really good choice, but more about that in a `second post <{filename}/mainline-kernel-galaxyS3.rst>`_.
  
 It's pretty easy, isn't it?
 ###########################
 
 Just get the newest kernel, build it for your target platform and flash it to the device. Unfortunately, it doesn't work like this in the case of phones. But if your laptop or PC is well supported by the Linux kernel, these are the only steps necessary to get the latest kernel working. Some GNU/Linux distributions even provide backports of recent kernel versions, but they normally add a few patches on top of the mainline kernel.
 
-The situation is completely different with practically all Android-based smartphones. Google maintains a `common Android kernel tree <https://android.googlesource.com/kernel/common/>`_ that contains lots of patches to make the kernel work with the Android user space. There is an ongoing effort to get these patches into the mainline kernel and the situation actually looks pretty good nowadays. You only need a few patches on top of the mainline kernel to be compatible with the most relevant parts of the Android user space. The common Android kernel is based on an older kernel version that has long-term support. So the only challenge is to get a few Android-specific patches working with the latest kernel or we could settle directly with the common Android kernel and only have to build it, right?
+The situation is completely different with practically all Android-based smartphones. Google maintains a `common Android kernel tree <https://android.googlesource.com/kernel/common/>`_ that contains lots of patches to make the kernel work with the Android user space. There is an ongoing effort to get these patches into the mainline kernel and the situation actually looks pretty good nowadays. You only need a few patches on top of the mainline kernel to be compatible with the most relevant parts of the Android user space. The common Android kernel is based on an older kernel version that has long-term support. So the only challenge is to get a few Android-specific patches working with the latest kernel or we could stay with the common Android kernel and only have to build it, right?
 
-Again, no, it doesn't work like this. If you are lucky, you will see some error messages on the screen and maybe a few other signs of life. The reason is that the phone manufacturers add a huge amount of out-of-tree code to the kernel to get support for their hardware. The added code mainly consists of drivers and support for the SoC. And as the manufacturers also base their changes on an old long-term support version at the beginning of development, the kernel that comes with the phone on release date is already pretty out-of-date. `1-3 million lines of out-of-tree code are normal for a smartphone <https://lwn.net/Articles/662147/>`_. So this is the point where it becomes really difficult to get a newer kernel working. And it is the reason why the manufacturers themselves don't bother updating their kernels to a newer version after the release.
+Again, no, it doesn't work like this. If you are lucky, you will see some error messages on the screen and maybe a few other signs of life. The reason is that the phone manufacturers add a huge amount of out-of-tree code to the kernel to get support for their hardware. The added code mainly consists of drivers and support for the SoC. And as the manufacturers also base their changes on an old long-term support kernel version at the beginning of the development, the kernel that comes with the phone on the release date is already pretty out-of-date. `1-3 million lines of out-of-tree code are normal for a smartphone <https://lwn.net/Articles/662147/>`_. So this is the point where it becomes really difficult to get a newer kernel working. And it is the reason why the manufacturers themselves don't bother updating their kernels to a newer version after the release.
 
 Kernel version 3.0 - make that old
 ##################################
 
-The latest source drop from Samsung for the Galaxy S3, that I am aware of, contains a kernel based on the 3.0.31 kernel version. This version was released on `7 May 2012 <https://lwn.net/Articles/496153/>`_. The phone became first available on `29 May 2012 <https://en.wikipedia.org/wiki/Galaxy_S3>`_. So besides some patches here and there, Samsung has probably never updated the kernel, not even to a newer minor release. CyanogenMod merged the latest release that is available for the the 3.0.x kernel series. This updated the kernel to the 3.0.101 version which was released on `22 Oct 2013 <https://lwn.net/Articles/571242/>`_. So the kernel is only a little more than three years behind. And by the way, I even found a few remains from the 2.6 kernel when poking through the source code.
+The latest released kernel source code from Samsung for the Galaxy S3, that I am aware of, contains a kernel based on the 3.0.31 kernel version. This version was released on `7 May 2012 <https://lwn.net/Articles/496153/>`_. The phone became first available on `29 May 2012 <https://en.wikipedia.org/wiki/Galaxy_S3>`_. So besides some patches here and there, Samsung has probably never updated the kernel, not even to a newer minor release. CyanogenMod merged the latest release that is available for the 3.0.x kernel series. This updated the kernel to the 3.0.101 version which was released on `22 Oct 2013 <https://lwn.net/Articles/571242/>`_. So the kernel is only a little more than three years behind. And by the way, I even found a few remains from the 2.6 kernel when poking through the source code.
 
-The difference in age is even worse when you want to port the out-of-tree code to a newer kernel version or if you want to merge a more recent kernel version because then the release date of the 3.0 kernel becomes relevant which is `21 Jul 2011 <https://lwn.net/Articles/452531/>`_, more than five years ago. The minor releases in the 3.0.x branch only include backports of important fixes from later versions. So the kernel doesn't include any of the substantial changes that were made to the Linux kernel in the last five years, except for a few that were backported by Samsung or CyanogenMod. The backports make it actually more difficult or outright impossible to merge a newer kernel, e.g. a 3.2.x kernel version, because some fixes were adapted for the older kernel and cause conflicts when merging a newer kernel.
+The difference in age is even worse when you want to port the out-of-tree code to a newer kernel version or if you want to merge a more recent kernel version because then the release date of the 3.0 kernel becomes relevant which is `21 Jul 2011 <https://lwn.net/Articles/452531/>`_, more than five years ago. The minor releases in the 3.0.x branch only include backports of important fixes from later versions. So the kernel doesn't include any of the substantial changes that were made to the Linux kernel in the last five years, except for a few changes that were backported by Samsung or CyanogenMod. The backports make it actually more difficult or outright impossible to merge a newer kernel, e.g. a 3.2.x kernel version, because some fixes were adapted for the older kernel and cause conflicts when merging a newer kernel.
 
 Merging a newer kernel
 ----------------------
 
-There is another reason that makes merging with newer kernel releases very difficult. A version control system like Git expects a common ancestor when merging two branches. But Samsung only releases archives of the source code without the Git history. They are not required by the GPLv2 licence to do this but it would be very helpful. Google publishes Git repositories of the kernels for their Nexus devices, but I am not aware of any project supporting Nexus devices that managed to merge major kernel releases, though.
+There is another reason that makes merging with newer kernel releases very difficult. A version control system like Git expects a common ancestor when merging two branches. But Samsung only releases archives of the source code without the Git history. Samsung is not required by the GPLv2 licence to release the Git history but it would be very helpful. Google publishes Git repositories of the kernels for their Nexus devices, but I am not aware of any project supporting Nexus devices that managed to merge major kernel releases.
 
-Without the Git history, Git has a very hard time to resolve conflicts when merging and you end up with an huge amount of conflicts, so many that it's just too much work to go through all of them. This happened when I merged with the 3.2.x mainline kernel tree. `I was able to get a partial merge working <https://code.fossencdi.org/kernel_samsung_smdk4412.git/log/?h=migrate_3.2>`_. The end result is not maintainable because there is too much unmerged code and you end up in the middle between two kernel versions. Merging more recent kernel versions becomes even more harder and you will end up with numerous bugs that no one else has. `Dorimanx targeted the Galaxy S2 and he did partial merges up to Linux 3.14 <https://github.com/dorimanx/Dorimanx-SG2-I9100-Kernel>`_, but he stopped working on it in 2014. The Galaxy S2 (GT-I9100) uses the same kernel sources.
+Without the Git history, Git has a very hard time to resolve conflicts when merging and you end up with a huge amount of conflicts so that it's just too much work to go through all of them. This happened when I merged with the 3.2.x mainline kernel tree. `I was able to get a partial merge working <https://code.fossencdi.org/kernel_samsung_smdk4412.git/log/?h=migrate_3.2>`_. The end result is not maintainable because there is too much unmerged code and you end up in the middle between two kernel versions. Merging more recent kernel versions becomes even more harder and you will end up with numerous bugs that no one else has. `Dorimanx targeted the Galaxy S2 and he did partial merges up to Linux 3.14 <https://github.com/dorimanx/Dorimanx-SG2-I9100-Kernel>`_, but he stopped working on it in 2014. The Galaxy S2 (GT-I9100) uses the same kernel sources.
 
 In the end, all of these efforts don't really bring a newer kernel to the device and we end up with an unmaintainable mess. The goal should be to have an easily maintainable kernel that makes future kernel updates possible. `In the second part <{filename}/mainline-kernel-galaxyS3.rst>`_, I will explain why going mainline is the only solution and how far I've got with getting the mainline kernel working.
diff --git a/content/u-boot-galaxyS3.rst b/content/u-boot-galaxyS3.rst
new file mode 100644
index 0000000..073ee38
--- /dev/null
+++ b/content/u-boot-galaxyS3.rst
@@ -0,0 +1,37 @@
+:title: U-Boot for the Galaxy S3: an investigation
+:date: 2016-11-26 21:19
+:tags: bootloader, kernel, free software, phone
+:category: General
+:author: Wolfgang Wiedmeyer
+:status: draft
+:summary: The Galaxy S3 comes with a bootloader named S-Boot which is a proprietary bootloader from Samsung. In this post, I want to collect information about the possibilty to run the bootloader U-Boot on the phone.
+
+The Galaxy S3 (GT-I9300) comes with a bootloader named S-Boot which is a proprietary bootloader from Samsung. In this post, I want to collect information about the possibilty to run the bootloader U-Boot on the phone. I personally didn't work on the bootloader part, but I became interested in it when `I tried to get the mainline kernel working and S-Boot prevented the mainline kernel from booting due to a bug <{filename}/mainline-kernel-galaxyS3.rst>`_.
+
+Benefits
+########
+
+`U-Boot is free software <http://www.denx.de/wiki/U-Boot/>`_. If you want to run as much free software as possible on your phone, then being able to replace the proprietary bootloader with a free one would obviously be great. A free bootloader would make it possible to control and verify which software is loaded during the boot process. In case of the Galaxy S3, it turns out that not only the Linux kernel and subsequently Android is loaded, but also a second proprietary operating system, a `TrustZone implementation called Mobicore <https://redmine.replicant.us/issues/1659#note-5>`_ and it's not fully clear what Mobicore does. A free bootloader could make it possible to decide if something like Mobicore should be loaded or not.
+
+A free bootloader would also make new security features like verified boot possible. The bootloader could check the signature of the kernel and the kernel could verify the integrity of the whole system. This way, it would be possible to ensure that no one and nothing has tampered with the operating system. More control over the boot process could also lead to new features that would make it a lot easier to unbrick devices that refuse to boot. For example an option that allows to boot from the SD card would be very helpful in this regard.
+
+Is it possible?
+###############
+
+In 2013, Dominik Marszk and Adam Outler managed to boot U-Boot from the SD card. Their U-Boot source can be found `here <https://github.com/Rebell/exynos4_uboot>`_. The Galaxy S3 tries to boot from eMMC (internal memory) first and if that fails, it attempts to boot from other possible boot devices like the SD card. In order to force booting from SD card, Adam corrupted the data transfer between CPU and eMMC by attaching a thin wire to one of the data lines that shortens the data line. After U-Boot has booted, it enters a 10 second delay to allow to release the line shortening. At this point, it's possible to talk to the U-Boot command line via `UART <{filename}/mainline-kernel-galaxyS3.rst#uart>`_.
+
+`These <https://github.com/Rebell/exynos4_uboot/tree/master/sd_fuse>`_ are the files they used to create the SD card. They come from `ODROID-X <http://www.hardkernel.com/main/products/prdt_info.php?g_code=G133999328931>`_ bootloader sources which are based on Exynos4412, just like the Galaxy S3. The binary p4412_s_fwbl1.bin is especially interesting because it is a first stage bootloader that drops the secure boot process and jumps to an unsigned payload. This signed first stage bootloader also works on the Galaxy S3 because the same signing key is fused into all Exynos4412-based development boards and handhelds.
+
+Dominik's and Adam's main problem was that they couldn't get a stock kernel image to boot. The cause was likely related to the TrustZone not getting initialized properly. They never tried a modified kernel, so this could be a place to start. `I disabled the Mobicore driver <https://code.fossencdi.org/kernel_samsung_smdk4412.git/commit/?h=replicant-6.0&id=7fbe662a46f3bb994b6f7a9adea731f3d8a5620c>`_ in the kernel for `Replicant 6.0 <https://blog.replicant.us/2016/08/replicant-6-early-work-upstream-work-and-f-droid-issue/>`_ in an attempt to get rid of any cooperation between the kernel and TrustZone. It would be interesting if it works with this kernel.
+
+The advantage of Dominik's and Adam's approach is the possibility to replace the proprietary bootloader S-Boot with U-Boot. A proprietary first stage bootloader is still necessary. And the second stage bootloader is created by a `proprietary tool mkbl2 <https://github.com/Rebell/exynos4_uboot/blob/master/mkbl2>`_ that is part of the U-Boot sources and was originally supplied by Samsung. The main disadvantage of their approach is the necessity to corrupt the data transfer between the CPU and eMMC which requires experience, is only suitable for development and testing purposes and has a high risk to render the device unusable if done wrong.
+
+There is some info floating around about a recovery procedure that boots from the SD card, but replaces the existing bootloader which resides in the bootloader partition on eMMC. `There are claims <http://forum.xda-developers.com/showpost.php?p=47234165&postcount=220>`_ that it's possible to trigger this process with more recent Galaxy S3 devices by pressing the menu key, both volume keys and the power key. Otherwise, it seems to be necessary to disassemble the device and shorten a very small resistor with a pair of tweezers, according to `leaked Samsung documents <https://smyl.es/samsung-galaxy-iii-s3-gt-i9300-jtag-leaked-document-how-to-repair-soft-bricked-galaxy-s3/>`_. This procedure could be an alternative to corrupting the data transfer between CPU and eMMC.
+
+As part of `my work with the mainline kernel on the Galaxy S3 <{filename}/mainline-kernel-galaxyS3.rst>`_, I submitted a `bootloader-related patch that allows to reboot the device in the recovery and download mode <https://patchwork.kernel.org/patch/9345815/>`_. In the discussion about the patch with the kernel maintainer Krzysztof Kozlowski, it turned out that Trats2, which is the Galaxy S3 released for Tizen, is supported by U-Boot. However, S-Boot was not replaced. Instead, U-Boot gets chainloaded after booting a (probably) adapted S-Boot image. `Krzysztof's mail <http://www.spinics.net/lists/arm-kernel/msg534042.html>`_ provides more info how the Trats2 image could be installed.
+
+Freedom-wise, chainloading U-Boot after S-Boot is not attractive as no non-free software is replaced. But having the Linux kernel loaded by U-Boot would make working on the mainline kernel easier because I don't have to figure out all the differences between U-Boot and S-Boot and patch the kernel. The only question in this regard is whether it's less work to stick with S-Boot and work around the differences or to get U-Boot for Trats2 working with a Galaxy S3 targeting Android and avoid having to patch the kernel.
+
+In the end, it would obviously be most rewarding if it's possible to replace S-Boot. If a signature check is enforced on S-Boot and the first and second stage bootloader are not replaceable, then there is no hope, unless the signing key gets leaked. If only the first and second stage bootloader need to be signed and S-Boot runs outside of the secure boot process, then maybe it's possible to get U-Boot working while the proprietary first and second stage bootloader stay in place. Maybe there is also some crazy way to make booting from SD card permanent while still having the internal memory accessible. So there are still some basic questions that need to be investigated.
+
+I want to thank Dominik Marszk, Adam Outler and Krzysztof Kozlowski for providing lots of useful information which made this post possible.