Flashing and hacking the android phone for Japanese Softbank network.

So, i went to Google Developer Day 2009 last tuesday and got free HTCMagic GDD2009 phone – limited edition.
It’s very cool toy, has everything i need, but as usual here, MMS/Mail does not work when using normal cellphone SIM card.

The Story:

Here, in Japan, people use E-mails to communicate (more popular than SMS), but these are not ‘real’ emails – these are MMS’s converted to emails on the fly by provider.
So, by sending email to addres xx@softbank.ne.jp, their server will find who is recipient and convert it to MMS, sending to this person.
Vice-versa, by sending email from phone, it actually is MMS that you send, and provider will convert it to e-mail outside the network.

The way MMS receiving works: special SMS comes to the phone, it contains first 1024 bytes of message and some data instructing it to download the rest – this remaining part is being downloaded via standard HTTP protocol, but this is when problem starts – Softbank blocks HTTP User-Agent of unknown phones (not being sold by them).

Another (and the most annoying) problem i encountered, is the android device itself, the edition i have, does not support sending MMS at all.

Solutions:

Step 1:

Make backup of your current device (you can skip it if you’re too lazy or confident ;-) ):

Then, once you have the file on SD card, reboot your android, and while it starts keep “BACK” and “POWER” pressed – android will enter fastboot mode, then type:

fastboot boot recovery-new.img

Your android will run this recovery image which allows you few things, like update from sdcard, make backup etc.

Choose the option to backup nandroid.
It will place backup of your current device onto the SD card – might be worth keeping :-).

YOU DO NOT need to ‘root the magic’ or do any other steps from this instructions – these steps are for other purposes, not applicable here.

—

Step 2:

Making android support MMS (if your provider does not block user agent this is the only and final step):

This will flash your device to the same version as Google IO event, you get few features like Voice synthesis (voice search and voice dial), Amazon MP3 access – these are not present in Japanese version.
But, the drawback is – You will not have Japanese language support anymore.

First, make sure you really need to enable MMS, go to Messaging application, start typing, and press menu, if you DO NOT have options lile ‘add subject’, ‘attach’ – then you DO need it.

You DO NOT need worry that instructions is for Google IO edition and not GDDC09 – these phones are in fact different but there are ways around it.

I recommend using fastboot method (its not at all hard),

You will need to downlaod fastboot binary (available at HTC), and 2 of these images: Radio Image and System Image.

Follow instructions for Radio Image installation, most likely your device will boot after it completes but you will get permanent android starting logo,
please have your USB cable connected and adb showing logging – you will see that it tries to boot all over again – THIS IS OK – no need to panic.

Place radio image onto your card by typing:

adb push <radio-image-package>.zip /sdcard/update.zip
adb shell sync

Boot into recovery mode by holding HOME while it reboots, Then choose ALT+s on the screen, next HOME+BACK, device should reboot, it might not start at this point by just showing you android logo – this is ok, no need to panic…

Once you have radio package downloaded and flashed , time to flash it with system image – this is little tricky:

Push the image onto SD card:adb push <system-image-package>.zip /sdcard/update.zip
adb shell syncBoot into Fastboot (BACK+POWER while you start it) – MAKE NOTE OF ‘hboot’ version – you will need it very shortly.

You can safely try to flash system image, but it will fail due to incompatible HBOOT version (they require 1.33.3004, and you have probably 1.33.3005).
No need to worry here, we can make it happen :-).

So, what you need to do, is to UNPACK your system image to a folder, edit the file android-info.txt, change the require version-bootloader to your hboot version (1.33.3005 most likely), save it, and compress all the files in this directory into new zip file (all of them need to be in the ‘root’ of zipfile).

So, now you have compatible system image that will work well on your device, try flashing it now, DO NOT FORGET to select options to clean userdata and cache – this is required, without it, your device will boot only to android logo and keep hanging there.

—

Step 3:

Modifying User-Agent string in the MMS client (this *MAY* be enough to complete only this step and it might as well make your device support MMS’s – i have done all steps though):
(GGDC09 version has MMS disabled, and i am not sure whether just changing MMS app is enough to enable this).

Further work: change more of android source, to have nice settings under MMS app, to edit UserAgent :-D.

Enjoy :-)

——-

UPDATE!!!

Thanks to Roberto Jung Drebes , now i know that there’s no need to flash GGDC09 with anything to get MMS working, all it takes is to use my MMS.app (or compile your own) – just the way i suspected.

Also, he found that setting apn type as ‘mms’ – doesn’t work unless you already have other apn with type ‘default’ – this way it works very well. Good way to ensure you don’t end up with huge phone bill due to data transfer…

for the over-lazy: there is an app in the market called handcentsms which enables you to set the user-agent.
I’m currently trying that out but haven’t found the correct APN-info so far… Kuba, couldn’t you help me out with the softbank mms settings? Thanks in advance

Here is my story:
I own an Android Dev Phone (currently running CyanogenMod 4.2.4) and I am using my Docomo sim card on it. I tried to use the APN settings from the HT-03 to connect to the BizHodai apn but was unsuccessful. When I try to connect to the non-mopera apn, it works fine (but that service is very expensive). The mopera-u apn simply refuses to work. If I’m not mistaken at first the mopera-u apn sends you some data but then stops working after the first few bytes.

I am not interested in SMS, only Data! Here are my questions:

1) How is Docomo blocking data access to non-docomo devices? Via IMEI? Is the UA string relevant to Data access or just MMS? i.e. Any chance for it to work on the mopera-u apn if I change my UA string?

2) Assuming the answer to 1) is no, Could I use a fixed-price data plan with the Android Dream if I move to softbank? I’m pretty sure softbank sell sim-card-only, I have asked them. If the APN setting change all the time, how will I know the new settings? Could I use my ADP instead of the magic?

3) If I DO use the magic with softbank after all, will I need to do this UA change if I need JUST data? (not mms)?

1) UA String matters only for MMS (but could also matter for http access depending on provider – but for that they would have to intercept all of your http traffic)
Do you have correct user/password etc for the mopera-u APN ? – try setting it up while having logging enabled via adb – it might hint you what is the rejection reason.
It will also help you to determine whether UA matters or not (wrong UA will most likely cause 403 error)

2)Yes, you can use iPhone card and iPhone data plan from softbank (i use it), but you will NOT be able to buy just the card with iphone data plan without buying iphone itself. The ordinary sim cards you can buy without a phone also provide data plan but it will cost you twice as much montly.
ADP will work the same as Magic.
3) No need to touch UA for data only, UA is ONLY for MMS (because it uses HTTP to send/receive messages).

4) I don’t think that’s possible, IMEI is usually hidden deeply in phone internal firmware – which is NOT the same as android itself. In some countries changing IMEI if possible is illegal – it’s equal to changing serial numbers on your cars engine etc etc.
Speaking of docomo blocking imei – yes, that is technically possible, but i don’t really think any operator would use it for that matter.
It’s being used for example to block stolen phones from working.

Thank you so much for the detailed information. So you think DOCOMO isn’t blocking me with IMEI (製造番号）? hmm… The manual for the HT-03 does not specify a username or a password for the APN settings (I have seen it with my eyes). You think some password exists? I have to investigate further. Thank you!

Hi, although blocking imei is technically possible, it’s pretty unlikely they would have gone that far – afterall you’re the one are making money on…
What docomo card are you trying to use ? maybe the card youre using just doesnt work with this apn –
softbank has special (black) cards for iphone and its unlimited data apn – which doesnt work on normal (silver) cards, the black ones also have some block which prevents using them on normal softbank phones.
Indeed there are no username/password for docomo apn, i will mail you settings i know (for docomo’s android card).
It is also possible that password mighe be hidden from showing to user – this should be possible to retrieve via backup etc.

Indeed my sim card might be unsuitable for that APN because I tried those APN settings (i thought my MCC/VC was wrong but the settings you sent me are the default after all and I have already tried with those settings. It doesn’t work). I might try again to debug it with the adb logger. Thanks again for all the help.

I’d like to ask, have you ever gotten message headers but when you download them, they become empty? I’m getting the message headers but when I download the message (manually – it doesn’t download automatically for some reason) a “Sender address hidden” notification shows up and the message appears to be from Anonymous with no message.

Do you mind telling me the exact settings you have to get it fully working?

Hi, unfortunately it does happen to me once in a while, i get mms, then when i download it , all content disappears,
i am able to reproduce this problem (for example by sending email from my corporate account), but simply didn’t bother and didn’t have time to investigate more on this – but i certainly will.

Thanks for the reply. Seems like I was able to receive from my Yahoo Japan email but not Gmail, Hotmail, and my friend’s Docomo mail.

So I decided to just open a new Gmail account for the phone and let people send emails to me through that. I hope the unlimited data plan (packet teigaku) kicks in and I won’t end up with a huge phone bill next month.

Just to be sure, as long as I set open softbank as the primary APN, I should be able to use the unlimited data plan, right? I do have a subscription for it.

Kuba, how do I contact you for the APN information? I just got charged over 1000 USD for five days of light internet browsing! I think I used the wrong APN info, so how do I do it the right way? I was on the flat rate packet discount, do I need to get on the iphone plan?

Quick question. Are you able to select to send MMS to contacts from the phonebook, or do you have to enter their mail address manually? Currently I have to enter the addresses manually and I was wondering if there was a way around this. I tried looking for some phonebooks that allowed this, and even MMS apps, but no luck so far.

I can send MMS to phonebook contacts and don’t understand why would this ever be a problem…

Wraithxt1, you don’t contact me :-), i contact you :-).
So, packet flat rate which you get with normal silver SIM card and ordinary phone DOES NOT guarantee you unlimited data transfer – even if it did, it would have cost twice as much as the same service with iPhone black SIM card.
I sent you mail with settings.
Softbank must have sent you warning SMS when you breached 30000 or 50000 JPY of the bill – if they haven’t you can even sue them.
Cheers.

Ah right.. but it will still send to the number, not the MMS address (eg. someone@softbank.ne.jp). It’s fine if I’m sending MMS to Softbank users (since I’m also on Softbank) but no go with other providers like Docomo.
Just to be clear, when you say you can send MMS, basically you’re talking about sending MMS to the recipient’s phone number, not their Softbank/Docomo/AU phone mail address, right? I think I might have misunderstood the whole MMS thing.

Thanks for the great guide Kuba!
I’m aaaaalmost there…
But MMS just doesn’t seem to be working.
I used the lazy method, and have tried all manner of User Agents, although I can’t find one for my specific phone (the Softbank X01T).
Using logcat, there seems to be an HTTP error (Forbidden) not long after the User Agent string is invoked.
Any tips?

Dzul1983 – Theoretically, MMS should work using just a number between operators, but i suspect they use normal e-mail protocol for this communication, so it may not work – i would have to send some to non-docomo people and see what happens.

Naginagi – UserAgent string has absolutely nothing to do with the phone, it is only between mms client and server, are you sure you entered correct UA ? (upper/lower case matters) – i think softbank only requires to see SoftBank at the beginning of the string, i will send you some UA samples later.

After lots of consultation with DOCOMO it appears that they indeed DO IMEI filtering on their BizHoudai service. No luck for my dev phone unless I can change the IMEI which, from what I’ve seen up to this point, looks impossible.

I just tried the 705NK UA and I can now send and receive to email addresses it seems :)
But sending or receiving to Japanese phone numbers (also Softbank) seems to force the mms app to crash!
The message I sent got through, but I couldn’t receive any of the messages.
I currently have the data and mms settings all in the same apn (named “default”) this isn’t connected by any chance is it?

Ok, everything seems to be working now!
I got this app from the Market called Handcent SMS… it made things a lot simpler. A huge improvement over the default MMS app, and it even has some pre-set UA’s to choose from!
Now it’s just a case of waiting for the next Softbank bill update to make sure my apn is for the right plan >_<
Thanks again for the great help!

Hi Kuba,
Sending MMS to different operators using the phone number doesn’t work in Japan. I’ve tried. However, if I manually insert the contact’s email address in the To field on the Messaging app, it sends the message just fine (provided it’s set up with the correct UA), and most messages that I get from my contacts show up fine (provided they use their phone mails eg. @docomo, @softbank, etc). The contact’s name shows up properly in the Messaging app to indicate who sent the message, and I can reply to this. Messages from web mails are kinda hit and miss. Gmail doesn’t work. Neither do Hotmail. Yahoo Japan mail works though.
The only thing I can’t do is select a contact from the Contacts app and send a message to their email using the Messaging app, and not Gmail.
I was thinking about modifying the Contacts.apk to allow sending a text message to an email address, but I’m a total noob to the android source and it would take some time to figure out stuff. Maybe you can point me in the right direction?

Still having various problems here… mostly similar to what otiasj seemed to be having.
Could somebody send me an example of working APN settings so I can see if there’s an error there?
Also when I reboot the system, the User Agent option disappears from the options menu and I have to adb install again every time… weird.

Yeah, I did, but it seemed to behave exactly the same as the proprietary messanger app.
For example, even if I set the user agent in the options menu it still only works if I install Kuba’s MMS.apk … but this seems to get reset every time I restart the handset :(
Sometimes I could receive, sometimes I could send, sometimes I couldn’t do either. I couldn’t isolate what was making it erratic.

Hello,
Could you please help me with setting MMS.
I am using HTC Hero. When trying to isntall the “lazy” mms.apk – I am getting an error saying – “42.0px could not be installed on this phone.”
Please let me know if someone knows a solutions to this.

PS. I have allowed non-market applications to be installed and there is 120Mb free on the Phone.

@Jubei, Changing IMEI is illegal in many countries, so be careful even if it was possible.

@naginagi, data and mms in one apn should not matter, if your app crashes, try using adb to figure out why (you should receive some error code when it does).
I didn’t have time to try and investigate why does the app disappear yet, but i will give it a shot soon.
(What i suspect at the moment, the app has been compiled on 1.5 sdk, if you’re using 1.6 maybe during startup it figures it out and overwrites the mms app.

@dzul1983, I think what operators did in Japan, is they couldn’t (or didn’t want to) agree on inter-operator MMS connections, so each had set up ms-email gateways which also gives advantage of interoperability with ‘normal’ email. This means there may be no way to send MMS between operators.
What i believe the phone’s MMS client does is when you select a contact it establishes connection to MMSC and submits MMS message using email address (of phone number if possible) of recipient.

@umka83, The MMS app i have doesn’t seem to work on Hero at all, and i don’t have proper source to compile it.
I’ve also tried running Hero on my Magic with no luck, so can’t work on anything Hero-related :( Sorry.
There should be a field for “apn type”, just enter “default” there.

@SpW, look into your mailbox, i’ve sent you the MMS settings you need.

P.S. Thanks :)

@ alohamark,
1. You will have to confgure APN on the unl**ked iPhone, it should work just the same way as we do it with Android.
2. Y let mE leave thiS without comment :P
3. Nope :P

Hi kuba!
I got my, non rooted, HTC Magic using prepaid softbank with Handcent (allows me to use custom User Agent) my apn for mms is correctly settled and I got MMS messages but just can read subject, can’t download it or send… any idea on how to fix this problem?
The phone I bought with USIM card is a 730SC.