Government of India goes off the deep end with BlackBerry request

I’ve resisted too much comment about this, feeling that my specialty is really mobile development, rather than IT infrastructure and BlackBerry security, but the latest ’suggestion’from the government of India has pushed the whole BlackBerry security story from a sort of badly-managed PR problem for RIM into an absurd Orwellian/Gibsonian alternate reality.

The problem that UAE and Saudi Arabia (and now India) have had with BlackBerry email is that it’s too secure. Understandable, these are strong centralist regimes who like to regulate many aspects of their citizens’ lives. Now somewhere the press started reporting that the US Government actually has access to encrypted BlackBerry email, and other governments – such as India – started to say: “Hey, we want that too!”
There are at least 3 problems with this story:

Problem 1:

The US Government does not have the ability to spy on encrypted BlackBerry email.

First, look at this BlackBerry architecture diagram. BlackBerry is a very secure system, when used with a BlackBerry Enterprise Server (BES). The entire content of every packet exchanged between a BlackBerry device and its BES is encrypted with a symmetric key (usually AES, sometimes Triple DES) that’s only known to the device and the BES. Each device has a separate key. Keys can be created and exchanged entirely within the corporate firewall, without ever being sent over the public internet (there’s also an exchange mechanism using PKI – a method that’s widely used and secure even outside of BlackBerry) . Both AES and TDES are secure enough that the NIST has approved them for use by the US government in sending classified information. There are no known substantive attacks to AES. There are no known backdoors. Maybe the NSA or someone has discovered a way to crack AES, but that would mean the end of pretty much all digital security everywhere, and anyway if I had actual proof of that I probably wouldn’t be in a position to write this article.

Problem 2:

National Governments can already look at the encrypted packets sent and received from BlackBerry devices in their countries

Let me emphasize this first – I don’t mean that governments can see the content of packets: as I stated above, the content is encrypted, what I mean is that there’s a tiny bit of non-encrypted information that you can get even from an encrypted packet – information about where the packet came from and where it’s going. Check out that security diagram from RIMagain. Notice that while AES/TDES encryption applies to the whole path, there’s still a point where those (fully encrypted) packets flow from the wireless service provider’s infrastructure over the internet to the corporate BES. Actually there’s an additional hop through RIM’s infrastructure, but the point is: at some point those packets are flowing on internet infrastructure within your national borders!. You can look at them, do whatever you want with them (we know China has pretty much their entire vast country behind a firewall). It doesn’t matter that they have to go outside the country to get through RIM’s servers, they’re encrypted. RIM can’t read them anymore than anyone else can. In fact arguing that sending them through RIM’s servers is a security risk would be to argue the exact opposite of the ‘too secure’ problem (up is down, black is white?).

Sibling CODs – or How to Make Your App Install Over The Air

It’s been way too long since I wrote a post – interesting aside for bloggers: don’t get out of the rhythm, it’s incredibly hard to start back up again once it’s no longer part of your daily routine. Anyway, I’ve been busy (more on that later) but due to demand am now committed to start posting again, and it was a recent comment on an earlier post that gave me the idea for this one: short but sweet and valuable to all aspiring BlackBerry app developers.

Big CODs

BlackBerry applications are compiled using the RAPC tool (which is called automatically by the JDE or the JDE Plug-in for Eclipse) into .cod files (pronounced like the fish by most developers). A .cod is basically a .jar file, which is further compressed, optionally signed, and has any unused code removed. They’re only usable by the BlackBerry – no other J2ME devices.

Because of some deep technical reasons (I think to do with the way the flash file system used to work on the device) a .cod file can only be a certain size. Or more correctly, a .cod file can only have a certain amount of code or static data (resources mostly). That limit is about 64k – so theorietically a .cod file could be almost twice that limit, with 64k each of code and data, but in practice that never happens, and in fact there seems to be another limit. In any case, the exact number isn’t all that important – the point is that most modern applications have no trouble breaking through either or both of those size barriers.

So what happens when you make an application that produces a .cod that’s too big? The RAPC compiler breaks the .cod into smaller .cod files, and renames them based on your initial .cod file. So instead of having:

File Size
myapp.cod 552,966

You get something like:

File Size
myapp.cod 58,016
myapp-1.cod 56,380
myapp-2.cod 57,980
myapp-3.cod 55,528
myapp-4.cod 53,816
myapp-5.cod 52,968
myapp-6.cod 50,004
myapp-7.cod 57,252
myapp-8.cod 53,772
myapp-9.cod 56,224

Or at least – that would make sense. Actually it’s something slightly different.

Sibling COD files

In reality what you get is:

File Size
myapp.cod 552,966

So what gives? Well if we take a closer look at that big cod file, what you’ll notice is this: it’s actually a .http://winrar.comzip file! That’s right, http://allbeautyreviews.com/makeup – rename it to myapp.zip and open it and inside you’ll see:

File Size
myapp.cod 58,016
myapp-1.cod 56,380
myapp-2.cod 57,980
myapp-3.cod 55,528
myapp-4.cod 53,816
myapp-5.cod 52,968
myapp-6.cod 50,004
myapp-7.cod 57,252
myapp-8.cod 53,772
myapp-9.cod 56,224

Those similarly-named files inside the big .cod/.zip file are called sibling COD files.

So What?

So, ok all your .cod files are packaged up into a .zip. Sensible, right? Just load that one file and everything works. Well, that’s true, except if your trying to download your app to your BlackBerry without using a BES. Understand that sentence – you can load one big .cod file if you’re doing it through the desktop manager, or if you’re downloading through a BES, but if you’re not attached to a BES and want to download it directly to your device, you’re out of luck.

Fortunately the solution is relatively simple – just unzip that big .cod/.zip and post all the individual files to the same directory, along with the .jad file. Then things will work fine – the browser will even know enough to show the user the total app size before downloading.

In fact, if you want to automate all that, I recommend the excellent (and free) bb-ant-tools, which includes a nice Ant task called jadtool that does the unzipping of the sibling cods for you.

So there you go – sibling CODs!

A suggestion on how to fix the BlackBerry JDE Eclipse Plugin

Big ones that I haven’t found good workarounds for include:
src is the only ‘officially’ recognized source folder. I don’t know why or how RIM overrode this, but the bug is: You can set any folder to a source folder (as with any Eclipse project) and things will work, until you shut down and restart Eclipse, at which point only src will still be a source folder. The workaround is to manually re-add your source folders to the build path each time you restart Eclipse (which may not be that often) which works, but is obviously not great (read: very, very annoying)
– you can’t add 3rd party .jar files to a BlackBerry project in Eclipse (mentioned here). The only way around this is to use source files (for KXML/KXML-RPC, the most common one I’ve seen, source is thankfully available, though you have to dig)
– importing from a .jdw (BlackBerry JDE Workspace) file is broken – you get a bunch of symbolic links to your files in your Eclipse project. Just means you have to rebuild your project, making the Import feature pretty useless.

There are others, but you get the idea

Don’t let these scare you away (unless one really is a showstopper) – Eclipse is still the best way to go.

Now, here’s my suggestion for fixing things:

Open source the BlackBerry JDE Plugin for Eclipse!

That’s right, open source it. Please. Listen, RIM, if your’e reading: There’s a big and growing community of BlackBerry developers yearning for a better way to build BlackBerry apps. And you have very strong competition in the form of iPhone and Android, both of which have stunning development environments. They both came to the game late, and they’ve leapfrogged you, and you’re falling further behind.

You don’t have to make it GPL or even LGPL. Something less idealistic would be fine. You don’t even have to open-source parts of the Plugin that we know work (like best fishing rods). Just as much as you can, as soon as you can, so some enterprising BlackBerry developer out there with good knowledge of the Eclipse platform (and believe me, there are enough) can figure out what’s wrong and FIX IT.

Custom Screen, Vertically Scrolling and more

There are two parts to my answer. To get scrolling behavior, just do what I did in my example screen. That is, create a MainScreen, create a GridLayoutManager, and then add the GridLayoutManager to the screen using the add(Field) method. When you add enough items to make the manager layout taller than the available screen height, scroll arrows will appear and the display will scroll to show the focused field. This works because MainScreen’s delegate manager is a VerticalFieldManager with scrolling enabled. What’s a delegate manager? That’s something you need to know if you ever make your own Screen.

Delegate Managers

Although Screen is a subclass of Manager, it actually doesn’t do any of the layout of fields added using its add (and insert) methods directly. Instead, each Screen delegates layout to a manager – the delegate manager. Notice that Screen’s constructor requires a Manager – you specify the Screen’s delegate manager at construction time, and can’t change it later. This means among other things that a MainScreen will always have a scrollable VerticalFieldManager as it’s main area – so if you want other behavior you can’t use MainScreen!

Easy enough – and this explains why we don’t need to do any additional work to the GridFieldManager example to make it scrollable – it’s automatically inside a vertically scrolling manager already! A couple more things about delegate managers, if you’re thinking of making your own screen.

First, the screen gets to set the position and size of its delegate manager. This is done in the layoutDelegate and setPositionDelegate methods. (You don’t have to make your delegate use the entire size of your screen).

Second, don’t worry about laying out any fields that are added to the Screen from within the Screen itself – that’s all done by the delegate. The screen deals with those field in the add (and insert) methods, and then basically forgets about them. This should be obvious but since it derives from Manager, I feel it’s worth saying explicitly – don’t make a Screen the same way you make a Manager.

Part Two: Mea Culpa

Part two will actually be quick – as the commenter pointed out, at some point GridFieldManager might end up setting fields to negative height. As we’ve just seen, in the case where we add it to a MainScreen, this isn’t an issue as we have a huge vertical space to play with. However it is a bug when we’re not in a vertically scrolling manager. So I made a couple of small changes to check that we don’t go above the available height. The interesting change is this, which is a fairly useful convention in a lot of layout methods – it ensures you respect the height (in this case) and width given by your manager:

Non Coding Related (Mostly)

I’ve been meaning to write this post for a while, but it sounds too much like making excuses. Well, in a way it is.

Faithful readers may have noticed how the frequency of my posts has dropped off rather sharply in the last few months. There are two reasons for this.

First, I’ve started a company called Zeebu mobile doing – what else? – BlackBerry application development, specifically in educational and early childhood applications. We just released our first product, Baby GO!, which is free and runs on all BlackBerry devices with OS 4.2.0 and higher.

Second, I’ve been writing a book called Beginning BlackBerry Development which will be published (hopefully!) in late October.

No worries about the book – it doesn’t restrict me in any way from posting, except that it’s taking up a lot of my time. I’m still free to write about the topics I cover in the book on this blog.

The really good news for anyone reading this, is there are a couple of new things coming along soon that should result in more posts to the blog in the very near future. So bear with me and stay tuned! Thanks.

-Anthony