As we speak Photo Resize is in the review queue at Apple, after a quick resubmission to fix some issues Apple didn’t like. I figured it would be valuable to others to talk about what went into the development of Photo Resize, and to go over lessons learned. Maybe someone at an earlier stage than me will read this and it will save them some time, but also it’s a good exercise to recollect what went right and wrong when a project wraps up.
Photo Resize came out of a day long brainstorming session between Mike and I in January. We generated a lot of ideas, and ranked, rated, debated and argued over the merits of all the ideas. Our goal was to find a small “doable” project that would get our teeth wet in app development, and get us experience in the submission process with Apple. From the initial design and our brief thoughts on how to build it Photo Resize fit the bill. Photo Resize would be an app that would allow you to send multiple images from your photo library via email to whoever you like. In addition it would resize and compress the images to save you bandwidth. This would allow you to send a bunch of images when you’re travelling and don’t want to pay expensive data roaming charges.
Design Phase
Mike and I hashed out an object diagram that broke down responsibilities between us. The design we settled on looked like this:
Don’t make fun of our design methodology! Show me how fancy yours is before you get all judgey. It served our purposes at the time. From this we were able to divide the work and walk away each man eager to sink his teeth into iPhone-code-a-mania.
Assumptions on the API we had at this time:
- We’d be able to grab query the photo library one image at a time as we like and get at image data with as much freedom as we liked.
- We’d have enough memory to select at least 25 or so images.
- We knew other apps were calling out to the iPhone mail app, so we figured we’d be able to as well. We didn’t know how good the support would be for attachments though, so this was a big question that needed to be answered right away.
Implementation
In the beginning my code responsibilities were the main screen, the image store (the manager that looked after all the selected images) and getting the images to the email application. Mike would take on the camera screen, magnified view and the image class that handled resizing and compression of the selected images.
Right off the bat we started running into problems:
- The API for getting pictures out of the Photo Library is extremely weak and very poorly thought out in my opinion. The only access you have to the user’s photo library is through UIImagePickerController, a class that unfortunately is a user interface you present to the user to let them pick a photo for your app to use for whatever purpose your app gets up to. This is extremely limiting, and really prevents the community from doing a ton of cool things with the user’s photo library in my opinion. I like that this class exists, if you’re making an app like “IfFound” then this component is great, does exactly what you need. If you’re trying to do some really advanced stuff or work with lots of images, this thing is a major wall in your way.
- Using the mailto: URL system to call out to the built in mail app was not possible with all the images. I tried a million different things. The most promising thing I tried was inlining the images as base64 uuencoded data, but nothing worked.
These two major issues forced us to rethink everything. Mike had to spend a great deal of time integrating the UIImagePickerController to get us access to the photos in a way that would work for us. I had to start researching what to do about emailing the photos. Luckily I came across SKPSMTPMessage. A very cool guy (Ian Baird) wrote a full working SMTP / email message class that would allow you to add as many attachments as you like. This saved the day for us. Without this Photo Resize probably wouldn’t exist, because I doubt we would have found it worthwhile to implement our own SMTP system. Being able to send the messages was great, but this opened up some new problems for us:
- How are we going to get the user’s email account settings?
- What are we going to do about writing out the message now that we aren’t going to use the built-in mail application?
Little did we know, these two questions would become serious hurdles to Photo Resize’s development.
Writing the message should be simple right? It’s just a UITableView with some UITextField’s in it. Psshht. Yeah right. It turns out Apple’s compose email screen is the most complicated damn thing on the planet. It has TONS of hidden functionality you don’t even notice when you use it, tons of things to make your life easier to pick people to send emails to. I had to duplicate all of this functionality. Things like the gray contacts table that comes up as you type and shows a filtered list of contacts based on what you’ve typed so far. Turning contacts into buttons… Adjusting controls in the cells as they grew and shrunk from addresses turning into buttons, getting deselected, turning into comma separated values. Bringing up the ABPeoplePickerNavigationController to let the user pick contacts from their address book, etc… The difficulty in making this screen work exactly the same as Apple’s screen just kept adding up and adding up. It took me 2 full weeks to completely duplicate the Apple screen. When it was done it was pretty sweet though.
The settings issue was contentious between Mike and I. I was of the opinion that this is a simple application, and we could simply just put all the settings for it in the settings bundle and tell the user when they start up to go there and fill out all their email account settings. Mike was opposed to this, and wanted to see functionality like Facebook’s app uses, where the first time you run the app it asks for all your settings before you can start doing what you do with the app. I knew this would be a lot of work to create a whole series of screens to get this information from the user. I didn’t want to do it, knowing how long it would take. But in the end, I agreed with Mike. We had to present a quality experience to the user, and we couldn’t just tell them to “go away and figure your settings out then come back.” So I started the process of duplicating everything the user sees when they set up the built in Apple mail application for their account. We wanted to support all the same providers Apple supports, so I set about duplicating every step you take when you go into the settings app and set up a new email account. Implementing all of this, hooking it up to the SKPSMTPMessage class and testing everything took another 2 weeks to get perfect. In the end it was for nothing though.
OS 3.0 and the bad news…
Midway through development Apple announced OS 3.0, and all the functionality it would include. This is the origin of an earlier post on this blog where I talk about Apple duplicating some of our functionality. It appears that in OS 3.0 the user will be able to send multiple images from the native photo app. That hurt. I felt being able to email multiple images was a major draw for this app and would warrant the $0.99 purchase all on its own, never mind the resizing and compressing to save bandwidth. Now the user will be able to do that with the built-in app, nobody’s going to use our app over a native app just for that. So now our hope lies entirely in the thought that people will want to save bandwidth when travelling. I think this is still a valid and useful thing, we’ll see what the public thinks when the app finally goes for sale.
This news, combined with the trials and tribulations we ran into developing Photo Resize brought Mike’s desire to be a part of Stormtap to an end. The financial stress, stress of choosing and developing the right product not knowing whether it would be successful or not, or whether at any time Apple might screw you, or the API might screw you was too much for him. He decided to return to a 9-5 job. So now I’m all by myself in Stormtap. That really sucked, took a bit to get over, but now I’m set and eager to keep going. Mike agreed to help me final Photo Resize and get it out there, and he might help out from time to time in the future on stuff I do too. We remain good friends.
Bringing it on home…
We had all of the major functionality implemented. You could enter your account settings, select a ton of photos, and email them all off to whoever you like from whatever address you liked. Everything was working great. In the simulator…
Mike started working off the phone and started hitting memory errors all the time. He started a full investigation and cleared up a lot of memory leaks and inefficiencies but we kept crashing running out of memory. After a great deal of investigation we found that the amount of memory you will have available to your app when it starts up has an extremely huge level of variance. You could have 70MB available to you when you boot up. Or you could have 10MB available to you. It all depends on what’s going on on the user’s device. Are they running iTunes? Did they fire up mobile Safari and open up a ton of pages? Have they left their phone running without rebooting for 3 months in a row? All of these things combine to a disaster memory environment for your app and you need to plan for these scenarios when designing an app. To solve all the memory issues once and for all I ended up implementing a tiered system. I would check the amount of memory available when the app loads and then set a variable limit on the number of images the user can select. When there’s lots of free memory we can go to 15 images safely. When there’s not much memory we drop down to a maximum of 5 images. This solved most of our problems. 20 – 25 images was too high a number anyways, the amount of data you were sending over 3G meant you’d be sitting there for quite a long time waiting for the email to send, and I feared running into timeouts with SMTP servers with this long of a process.
As I started really testing the app out I started running into all kinds of issues with various email accounts. Whenever I tested with my Gmail account I would randomly hit errors and be unable to send emails. The SKPSMTPMessage object would report that the server wasn’t available or that I failed to log in. It just seemed like randomly Gmail would go down or just fail to authorize accounts. When I tested Yahoo! we learned that only Yahoo! Mail Plus accounts are allowed to use SMTP (Yahoo! Mail Plus is a pay service), so I’m not sure what Apple does, maybe they have a deal worked out with Yahoo! to let the users send via SMTP even without Yahoo! Mail Plus. MobileMe pretty well. AOL was a complete disaster. It seems that AOL only lets you send over SMTP once you’ve verified yourself via a captcha. Annoying! So I started experimenting with alternatives. One of the things I tried was just replacing the from email with whatever email address and that worked. I tested this right at the beginning as a possible solution and it didn’t work, all I can think is that I must have had something set wrong that was the real reason it didn’t work. So now what our system does is uses the Stormtap SMTP account to send the email, but replaces the from address with an address given to us by the user at the start. That’s it. The system’s much more reliable, we are more certain it will work, we don’t have to worry about the user screwing up their settings (SSL, Validate SSL Chain, authentication method, etc… etc…) and it really simplifies the whole process. Unfortunately if we’d known this earlier we could have saved 2 weeks of time.
Submitting to Apple was fairly painless. I got it in and heard back from them in a couple of days. They didn’t like that we used the same button they use to start the email process from the photo app because we aren’t bringing up the action sheet like they do. Apparently you can only use that image if you bring up an action sheet. So I had to make a custom email button and redo all of the screenshots for the descriptions pages in all the different languages we support which was a major pain in the ass. They also complained that the app didn’t work well on the iPod Touch, even though I set that it was meant for iPhone only when submitting the app, which I found odd. It doesn’t work well on the iPod Touch because the touch doesn’t have a camera. So when you click the button in our app to bring up the camera bad stuff happens. To fix that I used the UIDevice class to find out whether we’re an iPhone or not, and if it wasn’t an iPhone I remove the camera button from the GUI. So now Photo Resize is in the queue and we’re waiting for it to be released to the masses!
What went right:
- Compressing and resizing the images was fairly simple, and does give true savings over the slight compression the native photo app does to images. ~60% drop in size without much quality loss.
- SKPSMTPMessage really saved the day. It’s very high quality, and Ian Baird is a real hero to us now.
- The way the main screen works held true to original design and really is a pleasure to use IMO.
What went wrong:
- Poor initial investigation led us into tons of unexpected work. We should have learned early on about UIImagePickerController, calling out to the mail app, and replacing the from email address in SKPSMTPMessage. Knowing all of these things ahead of time would have almost halved the development time and would have improved the overall design and implementation of the app. As it is now it’s stable but things are more like “crammed” into place and forced to work rather than seamlessly working together.
- Poor understanding of memory environment led to a lot of lost time and scrambling at the end. A more disciplined approach next time will save a ton of time finaling the next project.
- OS 3.0 being able to email multiple photos from the native photo app will likely be the nail in our app’s coffin. But we’ll see.
So this was a very long post, but I hope you’ve found it useful and that in some way it helps someone out in exchange for the tons of help I’ve found on people like Ian Baird’s blog.
It took 5 weeks but YAY!
With Photo Resize in review I’m all set to get the first app on the store and get this business rolling. I’ll be posting a large post mortem of the development of Photo Resize, but in the meantime if you’d like to learn about the project and what it looks like visit Stormtap’s website.
Rather than wait till the end to use Instruments to determine how many leaks you have I suggest that you build and tune/test your app in pieces. The current app we’re working on has tons of views and is very interconnected between the views. The next app I make will have some solid glue code, and the various views will be much more disposable. This will allow me to bring down the high watermark for memory, and also allow me to develop the app more carefully by tuning/testing each view at a time to make sure it has no leaks and no memory issues before moving on to the next thing. If you wait till the end it’s very difficult to figure out what’s going on in Instruments just because there’s SO much going on at once. It’s very difficult to filter out the “noise” of allocations that are OK from the bad allocations.
This is all common sense, and I’m sure when you read it you’re saying to yourself “Well duh. Retard.” Well, you get crunched for time and your good intentions slip away. So what I’m advising here is no matter how tempting it is to save time by cramming it in, you will save time by doing it right. I’ve learned now that this applies to all things Cocoa programming. If you don’t do things the “right” way, it’s very difficult to code your way out of the binds you get yourself into. Take the time to plan out your classes, your program’s flow and data model. No matter how much you hate it I promise it’s worth it.
Some day I know we’ll be free of the shackles of limited memory. Some day we’ll have unlimited pools that are completely unfillable. Until then we’re screwed.
The iPhone’s got 128MB of RAM (or so the docs say). Why does my app crash from out of memory errors and receive memory warnings when I only use ~15MB of that 128 then? I say because Apple lies. They just changed all their API functions to report big numbers when you ask for nifty specs. No the reality is just that for an OS that only lets one app run at a time, it turns out a lot of other stuff is running in the background hogging memory. If the user’s fired up 20 pages in mobile safari, there goes your memory. If your user needs to constantly rock out on the iPod while they use your fart app, there goes your memory.
In the future I’ll be designing all of my apps to run within a 20MB envelope. This should be plenty given my experience making apps run on the Nintendo DS’s measly 4MB. Time to fire up my code reloading, image compression, etc. skills.
It took me an entire evening to make these things so I could send them off to clubcard.ca to get made for me. Clubcard came recommended to me by Russ at Atomic Robot in Vancouver, hopefully they do a good job.
I went with the idea of having an iphone showing me as a contact, I think it turned out pretty good! Hopefully they turn out as well on the actual cards.
Still no word from Apple on our contract status.
Well if you’re reading my blog and interested in what we’re doing and feel like being a part of the action now’s your chance! All I need is a major credit card… Just kidding! Yeah just email me and we can arrange to get you hooked up. All you need is your very own shiny iPhone.
We stripped all the NSLog()’s out of our current app and the performance increase on the phone is very noticeable. I recommend you use something like a preprocessor macro you can #ifdef out.
If you’re having trouble figuring out where to set these preprocessor flags, do this:
- Select “Edit active target “<Your project name>”" from the Project menu.
- Set the configuration to the one you want it defined in.
- Scroll down till you see GCC 4.0 Preprocessing.
- Double click on “Preprocessor Macros”, add your stuff in there.
We’re very close to being finished our latest project, and by the end of the week we’ll be ready to upload it, depending on how difficult we find it to be to build adhoc builds to send out to friends to test for us. In the meantime we’re still waiting on Apple to approve our contract so we can get paid, it’s been 3 and a half weeks so far, every day twice daily I log into itunesconnect to be greeted by this:
Come on green checkmark!
