iQuit

As we speak Photo Resize is in the review queue at Apple, after a quick resubmission to fix some issues Apple didn’t like.  I figured it would be valuable to others to talk about what went into the development of Photo Resize, and to go over lessons learned.  Maybe someone at an earlier stage than me will read this and it will save them some time, but also it’s a good exercise to recollect what went right and wrong when a project wraps up.

Photo Resize came out of a day long brainstorming session between Mike and I in January.  We generated a lot of ideas, and ranked, rated, debated and argued over the merits of all the ideas.  Our goal was to find a small “doable” project that would get our teeth wet in app development, and get us experience in the submission process with Apple.  From the initial design and our brief thoughts on how to build it Photo Resize fit the bill.  Photo Resize would be an app that would allow you to send multiple images from your photo library via email to whoever you like.  In addition it would resize and compress the images to save you bandwidth.  This would allow you to send a bunch of images when you’re travelling and don’t want to pay expensive data roaming charges.

Design Phase

Mike and I hashed out an object diagram that broke down responsibilities between us.  The design we settled on looked like this:

Don’t make fun of our design methodology!  Show me how fancy yours is before you get all judgey.  It served our purposes at the time.  From this we were able to divide the work and walk away each man eager to sink his teeth into iPhone-code-a-mania.

Assumptions on the API we had at this time:

• We’d be able to grab query the photo library one image at a time as we like and get at image data with as much freedom as we liked.
• We’d have enough memory to select at least 25 or so images.
• We knew other apps were calling out to the iPhone mail app, so we figured we’d be able to as well.  We didn’t know how good the support would be for attachments though, so this was a big question that needed to be answered right away.

Implementation

In the beginning my code responsibilities were the main screen, the image store (the manager that looked after all the selected images) and getting the images to the email application.  Mike would take on the camera screen, magnified view and the image class that handled resizing and compression of the selected images.

Right off the bat we started running into problems:

• The API for getting pictures out of the Photo Library is extremely weak and very poorly thought out in my opinion.  The only access you have to the user’s photo library is through UIImagePickerController, a class that unfortunately is a user interface you present to the user to let them pick a photo for your app to use for whatever purpose your app gets up to.  This is extremely limiting, and really prevents the community from doing a ton of cool things with the user’s photo library in my opinion.  I like that this class exists, if you’re making an app like “IfFound” then this component is great, does exactly what you need.  If you’re trying to do some really advanced stuff or work with lots of images, this thing is a major wall in your way.
• Using the mailto: URL system to call out to the built in mail app was not possible with all the images.  I tried a million different things.  The most promising thing I tried was inlining the images as base64 uuencoded data, but nothing worked.

These two major issues forced us to rethink everything.  Mike had to spend a great deal of time integrating the UIImagePickerController to get us access to the photos in a way that would work for us.  I had to start researching what to do about emailing the photos.  Luckily I came across SKPSMTPMessage.  A very cool guy (Ian Baird) wrote a full working SMTP / email message class that would allow you to add as many attachments as you like.  This saved the day for us.  Without this Photo Resize probably wouldn’t exist, because I doubt we would have found it worthwhile to implement our own SMTP system.  Being able to send the messages was great, but this opened up some new problems for us:

• How are we going to get the user’s email account settings?
• What are we going to do about writing out the message now that we aren’t going to use the built-in mail application?

Little did we know, these two questions would become serious hurdles to Photo Resize’s development.

Writing the message should be simple right?  It’s just a UITableView with some UITextField’s in it.  Psshht.  Yeah right.  It turns out Apple’s compose email screen is the most complicated damn thing on the planet.  It has TONS of hidden functionality you don’t even notice when you use it, tons of things to make your life easier to pick people to send emails to.  I had to duplicate all of this functionality.  Things like the gray contacts table that comes up as you type and shows a filtered list of contacts based on what you’ve typed so far.  Turning contacts into buttons…  Adjusting controls in the cells as they grew and shrunk from addresses turning into buttons, getting deselected, turning into comma separated values.  Bringing up the ABPeoplePickerNavigationController to let the user pick contacts from their address book, etc…  The difficulty in making this screen work exactly the same as Apple’s screen just kept adding up and adding up.  It took me 2 full weeks to completely duplicate the Apple screen.  When it was done it was pretty sweet though.

The settings issue was contentious between Mike and I.  I was of the opinion that this is a simple application, and we could simply just put all the settings for it in the settings bundle and tell the user when they start up to go there and fill out all their email account settings.  Mike was opposed to this, and wanted to see functionality like Facebook’s app uses, where the first time you run the app it asks for all your settings before you can start doing what you do with the app.  I knew this would be a lot of work to create a whole series of screens to get this information from the user.  I didn’t want to do it, knowing how long it would take.  But in the end, I agreed with Mike.  We had to present a quality experience to the user, and we couldn’t just tell them to “go away and figure your settings out then come back.”  So I started the process of duplicating everything the user sees when they set up the built in Apple mail application for their account.  We wanted to support all the same providers Apple supports, so I set about duplicating every step you take when you go into the settings app and set up a new email account.  Implementing all of this, hooking it up to the SKPSMTPMessage class and testing everything took another 2 weeks to get perfect.  In the end it was for nothing though.

OS 3.0 and the bad news…

Midway through development Apple announced OS 3.0, and all the functionality it would include.  This is the origin of an earlier post on this blog where I talk about Apple duplicating some of our functionality.  It appears that in OS 3.0 the user will be able to send multiple images from the native photo app.  That hurt.  I felt being able to email multiple images was a major draw for this app and would warrant the $0.99 purchase all on its own, never mind the resizing and compressing to save bandwidth.  Now the user will be able to do that with the built-in app, nobody’s going to use our app over a native app just for that.  So now our hope lies entirely in the thought that people will want to save bandwidth when travelling.  I think this is still a valid and useful thing, we’ll see what the public thinks when the app finally goes for sale.

This news, combined with the trials and tribulations we ran into developing Photo Resize brought Mike’s desire to be a part of Stormtap to an end.  The financial stress, stress of choosing and developing the right product not knowing whether it would be successful or not, or whether at any time Apple might screw you, or the API might screw you was too much for him.  He decided to return to a 9-5 job.  So now I’m all by myself in Stormtap.  That really sucked, took a bit to get over, but now I’m set and eager to keep going.  Mike agreed to help me final Photo Resize and get it out there, and he might help out from time to time in the future on stuff I do too.  We remain good friends.

Bringing it on home…

We had all of the major functionality implemented.  You could enter your account settings, select a ton of photos, and email them all off to whoever you like from whatever address you liked.  Everything was working great.  In the simulator…

Mike started working off the phone and started hitting memory errors all the time.  He started a full investigation and cleared up a lot of memory leaks and inefficiencies but we kept crashing running out of memory.  After a great deal of investigation we found that the amount of memory you will have available to your app when it starts up has an extremely huge level of variance.  You could have 70MB available to you when you boot up.  Or you could have 10MB available to you.  It all depends on what’s going on on the user’s device.  Are they running iTunes?  Did they fire up mobile Safari and open up a ton of pages?  Have they left their phone running without rebooting for 3 months in a row?  All of these things combine to a disaster memory environment for your app and you need to plan for these scenarios when designing an app.  To solve all the memory issues once and for all I ended up implementing a tiered system.  I would check the amount of memory available when the app loads and then set a variable limit on the number of images the user can select.  When there’s lots of free memory we can go to 15 images safely.  When there’s not much memory we drop down to a maximum of 5 images.  This solved most of our problems.  20 – 25 images was too high a number anyways, the amount of data you were sending over 3G meant you’d be sitting there for quite a long time waiting for the email to send, and I feared running into timeouts with SMTP servers with this long of a process.

As I started really testing the app out I started running into all kinds of issues with various email accounts.  Whenever I tested with my Gmail account I would randomly hit errors and be unable to send emails.   The SKPSMTPMessage object would report that the server wasn’t available or that I failed to log in.  It just seemed like randomly Gmail would go down or just fail to authorize accounts.  When I tested Yahoo! we learned that only Yahoo! Mail Plus accounts are allowed to use SMTP (Yahoo! Mail Plus is a pay service), so I’m not sure what Apple does, maybe they have a deal worked out with Yahoo! to let the users send via SMTP even without Yahoo! Mail Plus.  MobileMe pretty well.  AOL was a complete disaster.  It seems that AOL only lets you send over SMTP once you’ve verified yourself via a captcha.  Annoying!  So I started experimenting with alternatives.  One of the things I tried was just replacing the from email with whatever email address and that worked.  I tested this right at the beginning as a possible solution and it didn’t work, all I can think is that I must have had something set wrong that was the real reason it didn’t work.  So now what our system does is uses the Stormtap SMTP account to send the email, but replaces the from address with an address given to us by the user at the start.  That’s it.  The system’s much more reliable, we are more certain it will work, we don’t have to worry about the user screwing up their settings (SSL, Validate SSL Chain, authentication method, etc… etc…) and it really simplifies the whole process.  Unfortunately if we’d known this earlier we could have saved 2 weeks of time.

Submitting to Apple was fairly painless.  I got it in and heard back from them in a couple of days.  They didn’t like that we used the same button they use to start the email process from the photo app because we aren’t bringing up the action sheet like they do.  Apparently you can only use that image if you bring up an action sheet.  So I had to make a custom email button and redo all of the screenshots for the descriptions pages in all the different languages we support which was a major pain in the ass.  They also complained that the app didn’t work well on the iPod Touch, even though I set that it was meant for iPhone only when submitting the app, which I found odd.  It doesn’t work well on the iPod Touch because the touch doesn’t have a camera.  So when you click the button in our app to bring up the camera bad stuff happens.  To fix that I used the UIDevice class to find out whether we’re an iPhone or not, and if it wasn’t an iPhone I remove the camera button from the GUI.  So now Photo Resize is in the queue and we’re waiting for it to be released to the masses!

What went right:

• Compressing and resizing the images was fairly simple, and does give true savings over the slight compression the native photo app does to images.  ~60% drop in size without much quality loss.
• SKPSMTPMessage really saved the day.  It’s very high quality, and Ian Baird is a real hero to us now.
• The way the main screen works held true to original design and really is a pleasure to use IMO.

What went wrong:

• Poor initial investigation led us into tons of unexpected work.  We should have learned early on about UIImagePickerController, calling out to the mail app, and replacing the from email address in SKPSMTPMessage.  Knowing all of these things ahead of time would have almost halved the development time and would have improved the overall design and implementation of the app.  As it is now it’s stable but things are more like “crammed” into place and forced to work rather than seamlessly working together.
• Poor understanding of memory environment led to a lot of lost time and scrambling at the end.  A more disciplined approach next time will save a ton of time finaling the next project.
• OS 3.0 being able to email multiple photos from the native photo app will likely be the nail in our app’s coffin.  But we’ll see.

So this was a very long post, but I hope you’ve found it useful and that in some way it helps someone out in exchange for the tons of help I’ve found on people like Ian Baird’s blog.