The NDK is a GCC-based cross-compiler that lets you embed object code in a traditional Android app. It also works as a standalone cross-compiler, though it is not advertised as such. It is quite rough when used by itself, and agcc wraps the complexity in a little script. While agcc and agcc2.pl are written in Perl, agcc.bash is a Bash script. It also supports the latest revisions of the NDK, though r7 is slightly buggy.

With agcc.bash, building native Android software feels almost like working with GCC. As a quick example, let’s try the classic hello world:

AGCC_NDK=~/android-ndk-r6 ./agcc.bash hello.c -o hello

That wasn’t too exciting… but what happened there? I have the NDK installed at ~/android-ndk-r6 and I told agcc.bash where to look for it, using the AGCC_NDK environment variable. I then invoked agcc.bash as if it were GCC. Let’s see what agcc.bash did behind the scenes:

AGCC_NDK=~/android-ndk-r6 AGCC_ECHO=yes ./agcc.bash hello.c -o hello

=> ./agcc.bash hello.c -o hello
<= /home/jyio/android-ndk-r6/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc -o hello -I/home/jyio/android-ndk-r6/platforms/android-8/arch-arm/usr/include -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ -DANDROID -DSK_RELEASE -DNDEBUG -UDEBUG -march=armv5te -mtune=xscale -msoft-float -mthumb-interwork -fpic -fno-exceptions -ffunction-sections -funwind-tables -fmessage-length=0 -march=armv5te -mtune=xscale -msoft-float -mthumb-interwork -fpic -fno-exceptions -ffunction-sections -funwind-tables -fmessage-length=0 hello.c -Bdynamic -Wl,-T,/home/jyio/android-ndk-r6/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/arm-linux-androideabi/lib/ldscripts/armelf_linux_eabi.x -Wl,-dynamic-linker,/system/bin/linker -Wl,–gc-sections -Wl,-z,nocopyreloc -Wl,–no-undefined -Wl,-rpath-link=/home/jyio/android-ndk-r6/platforms/android-8/arch-arm -L/home/jyio/android-ndk-r6/platforms/android-8/arch-arm/usr/lib -nostdlib /home/jyio/android-ndk-r6/platforms/android-8/arch-arm/usr/lib/crtend_android.o /home/jyio/android-ndk-r6/platforms/android-8/arch-arm/usr/lib/crtbegin_dynamic.o -lc /home/jyio/android-ndk-r6/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/lib/gcc/arm-linux-androideabi/4.4.3/libgcc.a -lm -ldl

See how it transformed our innocent little command into a monster! This might be overkill for a hello world program, but it provides pretty good support for more complex software such as OpenSSL, cURL, and Python. It would be a good idea to add the NDK toolchain and agcc.bash to $PATH before trying to build a big program. In my case, the toolchain resides in ~/android-ndk-r6/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin. If you want to build some software that come with configure scripts, you might start with:

CC=agcc.bash ./configure

Do try this at home! Relevant links:

• Android NDK r6 (or, the latest version)
• agcc.bash
• hello.c

CyanogenMod 7

The easiest thing to do is wait for a CyanogenMod ROM. Progress is delayed because Steve Kondik does not use an HTC Dream anymore and because of the lack of AOSP support. However, Mehdi Abaakouk has inherited Steve’s responsibilities and there will be a CyanogenMod 7 release sometime in the future.

For the impatient among us, other ROMs are floating around that might satisfy the desire to upgrade. Many of these solutions depend on the 2708/2825 radio and 1.33.0013d SPL for more efficient RAM utilization.

ezGingerbread

Maintained by ezterry, this near-AOSP source repository supplies Gingerbread for the Dream/Sapphire. It is primarily a source distribution with a number of unsupported binary builds.

EZ FRankenCYAN

Maintainer: bonesy
Heritage: Google, Cyanogen, Firerat, ezterry
Dependencies: radio and SPL update, CustomMTD for 94MB system partition

Ginger yoshi

Maintainer: yohshua
Heritage: bonesy
Dependencies: radio and SPL update
Features: apps2sdext, Google Experience apps, Zeam Launcher, MIUI Miren Browser, T-Mobile Theme Chooser

COS-DS

Maintainer: roalex
Heritage: Google, Cyanogen, Firerat, ezterry
Dependencies: radio and SPL update
Features: apps2sdext, Google Experience apps, ADWLauncher, T-Mobile Theme Chooser, Titanium Backup, Gallery 2D and 3D, ROM Manager, Google Maps 5 (vector maps, SD cache), S2E (simple2ext by OlegKrikun)

I am currently using Ginger yoshi, which performs quite well. Please share your experiences with running Android 2.3 on your HTC Dream/Sapphire. Also comment if anything is missing or inaccurate. Is it time for a hardware upgrade yet?

echo "src/gz inportb-android-froyo http://repo.inportb.com/android/froyo" > /cache/etc/opkg/inportb.conf

After doing that, we could refresh the package list

opkg update

Installing a package then becomes as easy as

opkg install name-of-package

And we could keep all our packages up-to-date using

opkg update; opkg upgrade

It’s also quite simple to set up a repository for distributing packages.

First, we fetch the latest opkg-utils off SVN at <http://svn.openmoko.org/trunk/src/host/opkg-utils/> and make it. Then, we designate a directory (the repository) to dump all our packages in. Finally, every time we update the repository, we generate a new package feed

cd path/to/repository; opkg-make-index . > Packages

Now all we have to do is make sure the directory is visible on the Web, and share its URL. The end user would add a line to a file within /cache/etc/opkg

src name-of-repository http://host/path/to/repository

We could also gzip the feed to reduce bandwidth usage

gzip -c9 Packages > Packages.gz

The repository line would be changed accordingly to

src/gz name-of-repository http://host/path/to/repository

We could have multiple files in /cache/etc/opkg, and each one of those could have multiple lines. The idea is that each vendor could have multiple repositories, and keep them in the same file for ease of management.

<ShamelessAdvertisement> Incidentally, it is almost trivial for Dropbox users to set up a repository. If you need 2.25GB of premium Web-accessible storage to host your repository or anything else for free, simply sign up through my referral link. You could also skip the referral and get 2GB.</ShamelessAdvertisement>

opkg install path/to/package.opk

or, if it’s on the Web

opkg install http://host/path/to/package.opk

And to remove the package, we would go

opkg remove name-of-package

But what if we wanted to share our own software with others? In this case, we would create our own packages. An Opkg package is essentially a Debian package with fewer control fields. If you know how to make a Debian package, you should be well on your way. In general, a package is an ar archive containing a control tarball, a data tarball, and a debian-binary file. For example let’s have a look at the opkg-hello package:

package.opk (ar)
+- debian-binary (text)
+- control.tar.gz (tarball)
...+- control (text)
...+- preinst (script)
...+- postinst (script)
...+- prerm (script)
...+- postrm (script)
+- data.tar.gz (tarball)
...+- system
......+- bin
.........+- opkg-hello (example program)

Now, debian-binary simply contains the string 2.0; it signifies that the archive is indeed a Debian[esque] package. The files within control.tar.gz are cached when the package is installed, and contain information about the package itself, how the package is to be installed, and how the package is to be removed. The contents of control describe the package and are used to build the repository feed, which we will cover later. There are also scripts that govern the installation and removal processes and require execution permission: preinst runs before the package is extracted, postinst performs any configuration after the main files are in place, prerm prepares the system before the package is removed, and postrm cleans up after the main files are gone. Let’s have a look inside these files:

control
Package: opkg-hello
Version: 0.0.1
Description: Sample OPKG package
Section: cyanogenmod/applications
Priority: optional
Maintainer: Jiang Yio
Architecture: all
Homepage: http://inportb.com/
Source:
Depends:

preinst
#!/system/bin/sh
echo "preinst: preparing to install package"
mount -o remount,rw -t yaffs2 /dev/block/mtdblock3 /system

postinst
#!/system/bin/sh
mount -o ro,remount -t yaffs2 /dev/block/mtdblock3 /system
echo "postinst: installed package"

prerm
#!/system/bin/sh
echo "prerm: preparing to remove package"
mount -o remount,rw -t yaffs2 /dev/block/mtdblock3 /system

postrm
#!/system/bin/sh
mount -o ro,remount -t yaffs2 /dev/block/mtdblock3 /system
echo "postrm: removed package"

Other than emitting some debugging information, the scripts also make the /system tree writable while the package is installed or removed.

Now that we have the anatomy of an Opkg package, we could create more like it with ease. Here are a couple of helpful commands:

make a tarball
tar zcvf name-of-tarball.tar.gz list of files and directories

make an archive
ar -r name-of-archive.opk list of files and directories

Other than that, just be sure all the files have the right owner, group, and permission. It would also help to know something about the Android directory tree — for example, we dropped the example program in /system/bin, which we know is part of the search path.

For user applications, we already have APK’s.

These are self-contained blocks that may be easily installed and removed. The APK system is great for running untrusted software that needs to be sandboxed, but it is not flexible or efficient enough to manage the operating system itself. Android users typically do not update their systems very often, and when they do so, they update the whole setup at once. This presents two problems: firstly, users don’t receive updated software as it becomes available and secondly, each update is a resource-intensive and time-consuming task. It would be great if system software could be updated piecemeal, just like a real Linux setup.

The Debian packaging system solves this problem very well, but its size makes it unsuitable for mobile devices. Nevermind that Cydia users use Debian packages — it’s still good to be frugal. Since we’re not going to install Debian software onto our phones (which would probably break many things), we don’t have to stick strictly to the Debian format; on the other hand, Debian provides a very nice model for us to emulate.

Enter Opkg.

Here is a package manager and format that very closely resemble the Debian package manager and format. In fact, Opkg is capable of working with Debian packages. The difference is that it’s tiny and it’s designed for mobile devices.

For starters, I have compiled a statically-linked build of Opkg. It is able to install local (and http-hosted) packages as dpkg does. It can also maintain a list of software sources a la apt-get. Its small size (745KB) makes it easily-integrable into almost any Android system.

If you’re feeling adventurous and would like to try this on your rooted Android phone, the files are currently hosted on my Dropbox account. You do need root access, so if you use a cooked ROM such as CyanogenMod, you should be good to go. Please keep in mind that there is only one package at the moment (the example), but it’s not difficult to make more.

For giggles, I decided to try viewing this page on my G1. I knew that the Android browser was pretty powerful, but I had no idea that it would handle such a complex application so well without modification…

So far, everything seems to be working fine with the exception of drag and drop events (I attribute this to the differences between the mouse and touch mechanisms). This is why standards are important, and why I continue to write compliant code.

While most users were playing around with the easy “apps2SD” feature, I decided to explore a different aspect of UnionFS — stackable snapshots. The goal was to be able to make snapshots of the filesystem that could be rolled back at any time, undoing any changes. While this may sound like server-oriented technology, it could be useful on a mobile platform that is collecting a sizable hacker community.

Here’s how I did it. I started with TheDudeOfLife’s latest build, which includes the new apps2SD trick from MarcusMaximus04. My microSD card has three partitions — the first is the usual FAT32 volume for documents, the second is an EXT2 volume for apps2SD, and the third is an additional EXT2 volume that I’m using to store my snapshots. If you follow the commands, you would see that I effectively bypass apps2SD for this experiment; but it is possible to use this method with apps2SD. Warning: the blogging software pretty-prints the quotation marks; be sure to make them regular double-quotes before copy+pasting.

1. Prepare a Mount Point

To mount the second EXT2 partition, we’ll need yet another mount point. Using ADB, remount the system partition in read-write mode:

adb remount

Next, get a shell session on the phone (through the terminal app or ADB) and type:

mkdir /system/sd2

2. Prepare the Snapshot Directories

We’ll be making three snapshots in this experiment, so let’s mount the second EXT2 partition and create some directories:

mount -t ext2 /dev/block/mmcblk0p3 /system/sd2
mkdir /system/sd2/snapshot0 /system/sd2/snapshot1 /system/sd2/snapshot2

3. Create a Snapshot

To create a snapshot, we overlay one of the snapshot directories over the existing data mount. But first, we must undo the original apps2SD overlay or this would not work:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
mount -t unionfs -o dirs=”/system/sd2/snapshot0=rw:/data=ro” none /data

As a test, let’s create a file:

touch /data/test0

4. Repeat

Now let’s create another snapshot and a new file:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
mount -t unionfs -o dirs=”/system/sd2/snapshot1=rw:/system/sd2/snapshot0=ro:/data=ro” none /data
touch /data/test1

5. Repeat

… and again:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
mount -t unionfs -o dirs=”/system/sd2/snapshot2=rw:/system/sd2/snapshot1=ro:/system/sd2/snapshot0=ro:/data=ro” none /data
touch /data/test2

6. Did it Work?

Let’s see. We have created three files, and we could check by listing them:

ls /data/test*

/data/test0
/data/test1
/data/test2

We could delete a file that we created in an earlier snapshot:

rm /data/test0
ls /data/test*

/data/test1
/data/test2

But when we switch to an earlier snapshot… not only was the most recently created file gone, but the deleted file came back:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
mount -t unionfs -o dirs=”/system/sd2/snapshot1=rw:/system/sd2/snapshot0=ro:/data=ro” none /data
ls /data/test*

/data/test0
/data/test1

As you can see, the last file we created just vanished. Now let’s roll back to the first snapshot:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
mount -t unionfs -o dirs=”/system/sd2/snapshot0=rw:/data=ro” none /data
ls /data/test*

/data/test0

There is just one test file now. And finally, let’s see what happens when we restore to the original mount:

mount -t yaffs2 -o rw,nosuid,nodev /dev/block/mtdblock5 /data
ls /data/test*

… the test files are all gone. But where have they gone? Why, they’re all on the snapshot partition, of course:

ls /system/sd2/snapshot*/

test0
test1
test2

7. Getting apps2SD Back

If you’re missing your apps2SD by now, you could do the following to get it back:

mount -t unionfs -o dirs=/system/sd:/data none /data

Or you could just reboot.

So…

Snapshots do work, though the procedure is a bit cumbersome. However, it should be trivial to write up a shell script to automate most of this. There are a couple of things that remain to be tested, however:

1. Performance: UnionFS is supposed to be very efficient, but I have not done any actual performance testing.
2. Efficiency: We created many overlay mounts back there while experimenting; I need to do some more reading to determine any impact this may have on efficiency.

What are the options?

As of now, there are three major builds containing this Android version. Haykuro maintains two series, the ‘G’ builds sourced from Google and the ‘H’ builds sourced from HTC for the Chunghwa phone. TheDudeOfLife maintains an analogous ‘G’ series. So far, there is no way to say which series is universally “better,” since each has its perks and quirks. So I would recommend trying them all if the time is available, since formatting is necessary anyway.

Where to get this Cupcake stuff?

Both Haykuro and TheDudeOfLife use Google Code to keep track of their progress:

• Haykuro’s builds
• TheDudeOfLife’s builds

They also post regularly at the xda-developers forum.

How to apply the update?

Haykuro has a nice guide written up. I found it quite helpful, but here’s some of my own commentary. Keep in mind that root access (specifically, the modified recovery image) is required. Haykuro’s guide covers that too, I believe.

What’s next?

For the adventurous (who isn’t who has performed this update?), funbacon has posted several enhancements.

• activation not required! (can set up account later)
• new theme
• new camera options, such as video recording
• lock-screen background
• 3G notification icon says H
• smooth transitions by default
• automatic screen orientation flipping
• no more voice search in search widget
• on-screen keyboard
• … among other things …

Now, it is actually pretty simple to get this software. There’s even a nice guide for this, which was pretty straightforward for me to follow. Nevertheless, since I had not rooted my phone and I have no data plan, I had a couple of extra steps.

0. Pre-requisites

It is necessary to format the microSD card as a FAT32 super-floppy. This means that there is no partition table and that there is a single FAT32 partition that spans the entire disk. Windows by default formats removable storage as super-floppies. In addition, it is necessary to get ADB working. These are out of the scope of this guide, but do ask in the comments if you’re having trouble.

Backup your data! This procedure will erase everything.

Before starting, please open up the Haykuro guide; what I write here only supplements the original information.

1. Downgrade to RC29 using DREAIMG.nbh

This was easy enough, but left me with a non-activated phone. To activate, I used JesusFreke’s trick. Since I had RC29, I entered setprop app.setupwizard.disable 1 on the keyboard to enable debugging, and raised the Settings manager by entering am start -a android.intent.action.MAIN -n com.android.settings/.Settings through the ADB shell.

2. Perform a nandroid backup

This is optional, but I did it anyway just to be safe.

3. Flash the new ROM

I downloaded the Chinese 5.0.1H_zh_signed.zip because the English version was not available. I didn’t mind because I know Chinese, but it was pretty easy to switch to English in the settings. For the record, the English-default version is now available for download.

Edit: I take that back; the English version just does not have Chinese fonts; it still needs to be switched to English. Therefore, I’d recommend getting the Chinese version.

3b. Switch to English

Now if you want to switch to English, start from the home screen and:

1. Push Menu
2. Touch the lower-right item (Settings)
3. Touch the next-to-last item (Locale & text)
4. Touch the first item (Select locale)
5. Select the first item (English)
6. Reboot for full effect

4. Upgrade the radio firmware

I did this because it was recommended, but I didn’t have any issues without it.

The factory data reset is not the best approach. In fact, I would say that the best approach is to do nothing at all about the problem. By virtue of Android’s memory manager, caches are cleared automatically when memory is needed. For example, if the foreground task needs to do something memory-intensive and there are some background tasks and caches, some of the tasks could be killed or caches could be cleared to make room. So… the problem solves itself.

One does not need to manually reclaim the memory, but what’s wrong with doing it anyway? I mean, it does get rid of the low-memory warnings that are so distressing to some. However, the biggest problem with clearing caches is the act itself. Caches perform the useful function of storing objects so that they need not be computed or fetched again, saving processor, networking, and power resources. One can even say that memory exists mainly for the purpose of caching. Therefore, it would be optimal to fill the memory as much as possible with cache. The system that Google set up is almost perfect — it allows the memory to fill with cached data, clearing bits at a time only when needed for a foreground task.

But why isn’t it perfect? Well, I had mentioned the low-memory warnings earlier. When the average user sees one of these, he/she thinks that the phone is about to explode. It’s not; rather, it’s a sign that the memory manager is doing its job. The psychological aspect of the problem can be resolved by either removing this warning or making it appear more benign, reserving the real warning for when the memory is really running out.

In the interim, I’m going to hang onto my caches and encourage you to do the same. There is no harm in taking the need-based memory management approach. Remember, unused memory is wasted memory.