The project involves quite a lot of thinking, to try to find the
optimal balance of security and usability, while staying within the
realms of what is possible on commodity phones.
For key establishment, PKI probably won't work, since X509
certificates are not available for individuals, and PGP seems too hard
for people to use. Moreover, it is not clear that phones have the
computational resources (and crypto libraries) for RSA cryptography.
Probably Diffie-Hellman key agreement is a better basis, with
authentication done either by face-to-face communication or by the
participants' ability to recognise each other's voices on the phone.
Storing keys or key material in the contacts list (in the place of
another number) may be useful. A symmetric encryption algorithm such as
AES (or even 3DES) should work fine.