NXT-Spy – First successful data transfer to a remote computer over Wi-Fi on Android 1.5

After some research on Google, I have finally found an article about Sockets that shows how to send requests between a client and a server located on the same network [source].

So I decided to implement it to do a little conversation between the client and the server. Obviously, the phone will be the server and the PC the client because it will be the PC that will be requesting an image to the phone.
So I implemented my ServerSocket thread on the phone and a ClientSocket on the PC but it fails : the connection is not opened and nothing happens. But after some research, I finally found the solution [source].

I now have a working dialog between the phone and the PC. It’s now time to send an image from the camera instead of just some text, and this is where it is getting tricky.

After some research, I find how to get the bytes of the current preview image thanks to a Callback function [source]. So I thought that everything was solved : I had a byte array corresponding to an image, and my connection class was made to send byte arrays! So I managed to put all the things together quite quickly, created a simple GUI on the PC in order to be able to display a picture on the client and finally initiated the connection.

The connection works, the bytes are transferred but the client just won’t rebuild the image from them. After a lot of research, I finally finds the beginning of solution here [source, source]. In fact, event if we set the preview format to JPEG, the format of the bytes provided by Android camera PreviewCallback function are always in YUV format. So to solve the problem, we have to add convert the frames from YUV to RGB, encode them in JPEG, and finally convert them to a byte array. [source]. (Note : this solution is quite heavy for a phone, I should perhaps convert the data on the client instead)

I now thought that everything was solved now but no! Still not : I have some bitmap transferred the right way and I thought in the right format but on the computer screen, I get an unclear and stripped image.

So now I am getting quite annoyed by this callback function that seems to be completely buggy. So decide to change my approach by instead of getting the image from the preview callback, getting it from the takePicture callback (every time we take a picture). So I commented my YUV decoding solution and send directly the bytes obtained from the takePicture callback. Finally, did a perpetual loop that takes picture continuously.

And it is a success! I finally have my first successful image transfer from the phone’s camera to the computer screen over Wi-Fi. But I still have one major problem : I have a frame-rate of something like 1 image every 10 seconds.

NXT-Spy – First tests with the HTC Hero

After I installed and configured the Android SDK and synchronized it with Eclipse [source], I am starting to code my first Android application.

I am creating a new project, it is already pre-configured with a simple HelloWorld project. I discover that the components of the GUI are created via a XML file called main.xml, so to add a component, we just have to add a XML tags [source]. Then, to access them from the main Java class, we just have to get the elements from their ID.

So,  to create my own application, I have first drawn some sketches of the screen on an application called Pencil [source].
So for the Wi-Fi connection indicator and the transfer status indicator, I am using a ImageView component. For the Camera preview zone, I am using a SurfaceView (because the camera service needs a its holder to display the preview). Finally, for the IP and SSID display, I am using a TextView.

Now that my GUI is done, I am soon going to start working on the Java code.

NXT-Spy – Which phone will I use ?

Because I love to have¬† a little computer in my pocket, It turns out that to possess three different smart-phones that are equipped with both Wi-Fi and Bluetooth : An old Qtek 9100 of 2005 (Qtek’s version of the HTC Wizard), a Nokia N900 that I bought last year which is my current personal phone and a brand new HTC Hero that I’ve been given by Orange because I changed my telephone subscription.

So here is a little comparison of the three to decide which one I am going to use for this project :

So, to conclude, I won’t use the Qtek 9100 (for the moment at least) because it won’t allow me to have access to a complete enough JVM and because it’s way less powerful than the two others.

For the two others, it’s harder to tell which one is the most appropriate as the two devices are amazingly powerful and Linux based :
* The HTC Hero has a very powerful JVM that includes “android” classes that will give me a really easy access to the device’s functionality (like the camera) and even if it’s not a genuine one, it should definitely do the trick. I also really like the Android SDK running on Eclipse.
* The Nokia N900 is more interesting in the fact that it is completely open and very powerful : I will be able to install a full JVM and even run Groovy scripts. But there are two major problems with this one : Because Java support is not native, I will probably have some troubles to have access to the phone’s camera from Java. The second problem is that the SDK is not compatible with Java programming and is really hard to install and to use.

So to get started in the project, I will use the HTC Hero and code the program in Java with the Eclipse SDK. Further in the project, I will perhaps go back on the Nokia N900 to try to code my application in Groovy parallel.