Anyone who has attempted to perform a large HTTP file upload from a memory constrained device will quickly discover serious roadblocks.  Using the HTTPBodyStream property of an NSMutableURLConnection instead of HTTPBody is simple enough in theory, but it’s not a s simple as sending a single file.  There are HTTP headers and query string parameters and cookies and multipart encoding strings and boundary identifiers and all sorts of other things that need to be written in front of, behind, and around the data that you are going to upload, so what kind of input stream can we use?

Apple provides three options, but none of them work very well for our needs.  Creating an NSInputStream from an NSData buys us nothing, since that works in the same way as using HTTPBody.  Creating one from a file path is a potential option, by writing out all the headers and other parts to a temporary file and then handing it all to the NSMutableURLConnection as a single file stream, but this is incredibly inefficient.  Creating a low level BSD socket and reading and writing from a pair of socket streams is a slightly less inefficient option, but is prone to the dangers of using darwin-level API calls that Apple makes no guarantees about.

The obvious solution, of course, is to create your own subclass of NSInputStream that handles both blocks of data and sub-streams that point to underlying file system data, so let’s try that here:

So now you have a multiplexed input stream, and in all your local testing it works great.  So, it’s ready for prime-time, right?  Whoops, not quite.  You’ll quickly discover that mysterious, undocumented calls are being made on your subclass that you haven’t implemented or have even even heard of, namely _setCFClientFlags:callback:context: and _scheduleInCFRunLoop:forMode: and that’s crashing your code.  What’s going on here?

First a quick note about NSStreams: they are not self-propelled, at least not all of them.  For example, if you are writing out to a file, do you want each and every byte written to the disk individually?  Or is it better to write to a memory buffer and then flush the buffer periodically?  The latter is much more efficient, but who is in charge of the flushing operations?   NSStreams answer this question by leaving the option up to the caller by providing a way to supply a runloop that the stream can use to perform it’s operations.

If you implemented your new NSStream to the documentation, you’ll have seen that scheduleInRunLoop:forMode: is required to create a proper NSStream subclass.  So why is that method not being called, but this undocumented _scheduleInCFRunLoop:forMode: call being made instead?  This has to do with “Toll Free Bridging”.

Although there are many NS classes that are perfectly “toll free” bridged to their CF counterparts, NSRunLoop is not one of them.   CFRunLoop has a similar name, but these two are not compatible to switch between.  Instead of calling the NSRunLoop methods that you implemented, the NSMutableURLConnection is trying to use the CF equivalents, methods that are undocumented.

It turns out that working around this is more trivial than it seems once you understand what is going on.  The NSInputStreams that reads from NSData and file handles are perfectly capable of managing their own read operations without an external runloop, and since we plan to be multiplexing only those types of input to create our subclass, these two calls can be safely ignored altogether.

We still need them to be answered in our class when the URLConnection calls them, and we need to try to avoid getting caught up in any automatic “undocumented API” filters in the App Store submission.  Let’s do that by not having those methods implemented directly, but instead have resolveInstanceMethod try to forward any undocumented calls to methods that don’t directly override undocumented APIs:

Simply add these lines to the class above and presto, now you have a fully functional NSInputStream that multiplexes substreams and allows you to upload any number of large files without ruining your application’s memory profile.