Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Image Processing and Computer Vision in iPhone and iPad

23 777 vues

Publié le

Mini-course presented at WVC 2011, Curitiba (Brazil), May 22, 2011.

Publié dans : Technologie
  • Protect your brain from memory loss with brain pill. find out more... ♥♥♥ https://tinyurl.com/brainpill101
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Is Your Ex With a Man? Don't lose your Ex girlfriend! This weird trick will get her back! ♥♥♥ http://ishbv.com/exback123/pdf
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT, ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • ★★ How Long Does She Want You to Last? ★★ A recent study proved that the average man lasts just 2-5 minutes in bed (during intercourse). The study also showed that many women need at least 7-10 minutes of intercourse to reach "The Big O" - and, worse still... 30% of women never get there during intercourse. Clearly, most men are NOT fulfilling there women's needs in bed. Now, as I've said many times - how long you can last is no guarantee of being a GREAT LOVER. But, not being able to last 20, 30 minutes or more, is definitely a sign that you're not going to "set your woman's world on fire" between the sheets. Question is: "What can you do to last longer?" Well, one of the best recommendations I can give you today is to read THIS report. In it, you'll discover a detailed guide to an Ancient Taoist Thrusting Technique that can help any man to last much longer in bed. I can vouch 100% for the technique because my husband has been using it for years :) Here's the link to the report ■■■ https://tinyurl.com/rockhardxxx
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • hi sir ,
    Can you please mail me this presentation it would be helpful to me

    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Image Processing and Computer Vision in iPhone and iPad

  1. 1. Image  Processing  and  Computer   Vision  in  iPhone  and  iPad   Oge Marques, PhD omarques@fau.edu VII Workshop de Visão Computacional (WVC) Curitiba, PR - Brazil May 2011
  2. 2. Take-home message •  Mobile image processing and computer vision applications are coming of age. •  In this mini-course you will learn suggestions, ideas, and resources for developing iPhone and iPad apps in this field.
  3. 3. Disclaimer •  I’m a teacher, researcher, graduate advisor, author, … •  … not a developer
  4. 4. Background and Motivation Two  sides  of  the  coin  •  The maturity and •  The unprecedented success popularity of image of the iPhone, iPod Touch, processing and computer and – more recently – the vision techniques and iPad algorithms
  5. 5. Motivation •  Rich capabilities of iPhone for image and video processing •  Apple support for image and multimedia: frameworks, libraries, etc. •  Third-party support for iPhone-based development: open APIs, OpenCV, etc. •  Success stories and ever-growing market
  6. 6. iPhone 4 technical specifications 1  GHz  ARM  Cortex-­‐A8   processor,  PowerVR   SGX535GPU,    Apple  A4   chipset  
  7. 7. iPhone photo apps •  Huge growth in mobile apps, including image-rich iPhone apps: –  200+ photography-related apps available in iTunes store –  Entire sites for reviews, discussions, etc. –  Subcategories include: •  Camera enhancements •  Image editing and processing (yes, Photoshop on the phone!) •  Image sharing (Instagram: 1 M users in less than 2 months!) •  Image printing, wireless transfer, etc.
  8. 8. Smart phone market
  9. 9. Smart phone market Source: http://www.cellular-news.com/story/48647.php?s=h
  10. 10. Smart phone market •  According to the American Academy of Cosmetic Dentistry: –  5.1 billion people have a cell phone –  4.2 billion people have a toothbrush Source: http://www.softcity.com/contribution/AO4UTMwADN/view/more-cell-phone-owners-than-toothbrush-owners
  11. 11. Outline •  Part I – Fundamentals of iOS development – Core concepts, techniques, and tools •  Part II – OpenCV and iOS – Functionality and portability issues •  Part III – Examples and case studies – Commercial apps and students’ projects
  12. 12. Part IFundamentals of iOS development
  13. 13. Brief survey •  Have worked with Object Oriented Programming? •  Have installed the iOS SDK? •  Have developed apps for iPhone / iPod touch / iPad? •  Have submitted apps to the App Store? •  Have developed image processing / computer vision software? •  Have worked with OpenCV?
  14. 14. Checklist •  What you’ll need: –  Intel-based Macintosh running Snow Leopard (OS X 10.6.5 or later) –  Sign up to become a registered iOS developer •  Apple requires this step before you’re allowed to download the iOS SDK and many other goodies from the iOS Dev Center –  xCode 4 and latest (4.3) version of iOS SDK –  iOS simulator –  iPhone, iPod Touch, or iPad (optional)
  15. 15. Welcome to the jungle •  Only one active app •  Only one (fixed-size) window •  Limited access (“sandbox”) •  Limited response time •  Limited screen size –  640 × 960: retina display devices (iPhone 4, 4th-gen iPod touch) –  320 × 480: older devices –  1024 × 768: iPad •  Limited system resources •  No garbage collection •  No (physical) keyboard or mouse
  16. 16. App Bundle •  Your code •  Any frameworks you’re linking to •  Nib files (interface builder) •  Resources (images, sound, etc.) •  PLIST files (app settings)
  17. 17. Tools and documentation •  Xcode •  Interface Builder •  iOS Simulator •  Instruments •  Apple documentation
  18. 18. Xcode 4: what you need to know •  Workspace window •  How to open an existing project •  How to navigate through the project’s contents •  How to organize views, subwindows, etc. •  How to use the debugging options •  How to create a new project •  How to edit targets and build settings •  Taking advantage of code completion and other sources of immediate help •  How to find help and integrate with Apple documentation
  19. 19. Xcode 4 •  Workspace window
  20. 20. Interface Builder •  Workspace window •  How to create a new NIB file •  How to add views and objects to the UI •  How to make connections between IB objects and source code
  21. 21. Interface Builder •  Workspace window
  22. 22. iOS Simulator •  How it integrates with Xcode •  What it can do –  Simulate different devices and generations (iPhone before and after retina display, iPad) –  Basic device behavior (home button, etc.) –  Selected built-in apps (Safari, Contacts, etc.) –  Simulate rotation, low memory warnings, etc. •  What it cannot do –  Provide a good measure of app performance on the actual device –  Simulate camera, accelerometer, etc.
  23. 23. Instruments •  Very rich tool (integrated with Xcode) for app performance and behavior evaluation
  24. 24. Example •  Hello, World!
  25. 25. iOS technology layers Our primary interest Most iOS courses, books, etc. (2D/3D graphics, image, audio, video)
  26. 26. Object Oriented Programming •  Class vs. Instance •  Methods •  Instance Variables (properties)
  27. 27. 3 Pillars of OOP •  Encapsulation - hide the details from the outside world •  Polymorphism - different objects, same interface •  Inheritance - acquire the features of the parent and add onto its capabilities
  28. 28. Inheritance in Cocoa Touch
  29. 29. What is Objective C? •  Superset of C –  Mixing ObjC and C, or even ObjC and C++ –  Syntax a little different than C –  Strong or loosely typed (your choice)
  30. 30. Additions from C •  Types –  Anonymous object –  Class –  Selectors •  Class definition •  Sending messages
  31. 31. File extensions •  Objective-C is a superset of ANSI C and supports the same basic syntax as C. •  Objective-C files use the extensions below: Based on “Learning Objective-C: A Primer”
  32. 32. #import vs. #include•  When you want to include header files in your source code, you typically use a #import directive. •  This is like #include, except that it makes sure that the same file is never included more than once. Based on “Learning Objective-C: A Primer”
  33. 33. Classes Instances •  A class is... –  a blueprint to creating an instance –  an object too (type Class )
  34. 34. Classes Objects •  Classes declare state and behavior for a type •  An instance of a class maintains its state through instance variables •  Behavior is implemented through methods •  Instance variables (iVars) are typically hidden –  Usually accessible via getter and setter methods, or the handy property
  35. 35. Classes in ObjC •  Specification of a class in ObjC: –  Interface: class declaration, instance variables (ivars), and methods associated with the class. Usually a .h file. –  Implementation: actual code for the methods of the class. Usually a .m file. Based on “Learning Objective-C: A Primer”
  36. 36. Class declaration Based on “Learning Objective-C: A Primer”
  37. 37. Header File •  Example:
  38. 38. Implementation File •  All implementation goes between @implementation and @end •  use #pragma mark to block groups of code together
  39. 39. Strong and weak types •  ObjC supports both strong and weak typing for variables containing objects. –  Strongly typed variables include the class name in the variable type declaration. –  Weakly typed variables use the type id for the object instead. Typically used for things such as collection classes, where the exact type of the objects in a collection may be unknown. Based on “Learning Objective-C: A Primer”
  40. 40. Instances, methods, and messaging •  A class in ObjC can declare two types of methods: –  instance methods: execution is scoped to a particular instance of the class. •  Before you call an instance method, you must first create an instance of the class. –  class methods: do not require you to create an instance. •  These methods must be called on the class. Based on “Learning Objective-C: A Primer”
  41. 41. Instances, methods, and messaging •  Method declaration syntax Based on “Learning Objective-C: A Primer”
  42. 42. Instances, methods, and messaging •  When you want to call a method, you do so by messaging an object. •  A message is the method signature, along with the parameter information the method needs. •  All messages you send to an object are dispatched dynamically, thus facilitating the polymorphic behavior of Objective-C classes. Based on “Learning Objective-C: A Primer”
  43. 43. Messaging •  When you call a method, its referred to as sending that message to the receiving object. •  Message Expression [receiver method:argument]; •  Example: [myCar accelerate]; will send the accelerate message to object myCar
  44. 44. Message (cont.) •  Named arguments [myCar turnOnRadio]; [myCar turnOnRadioWithStation:93.1]; [myCar turnOnRadioWithStation:93.1 volume:0.8]; •  Definitions of these methods: - (void)turnOnRadio; - (void)turnOnRadioWithStation:(double)station; - (void)turnOnRadioWithStation:(double)station volume:(double)volume;
  45. 45. Instances, methods, and messaging •  Messaging example: –  To send the insertObject:atIndex: message to an object in the myArray variable, you would use the following syntax: [myArray insertObject:anObject atIndex:0]; Based on “Learning Objective-C: A Primer”
  46. 46. Instances, methods, and messaging •  Messages can be nested. –  Example: •  If you had another object called myAppObject that had methods for accessing the array object and the object to insert into the array, you could write the preceding example to look something like the following: [[myAppObject theArray] insertObject:[myAppObject objectToInsert] atIndex:0]; Based on “Learning Objective-C: A Primer”
  47. 47. Instances, methods, and messaging •  ObjC also provides a dot syntax for invoking accessor/mutator (getter/setter) methods. •  Using dot syntax, you could rewrite the previous example as: [myAppObject.theArray insertObject: [myAppObject objectToInsert] atIndex:0]; Based on “Learning Objective-C: A Primer”
  48. 48. Instances, methods, and messaging •  You can also use dot syntax for assignment. •  Example: myAppObject.theArray = aNewArray; is simply a different syntax for writing [myAppObject setTheArray:aNewArray]; Based on “Learning Objective-C: A Primer”
  49. 49. Instances, methods, and messaging •  To use the dot syntax, your getter and setter methods must have a specific method signature: - (NSString*)myString; //getter - (void)setMyString:(NSString*)aString //setter
  50. 50. Instances, methods, and messaging •  Class methods –  Typically used as factory methods to create new instances of the class or for accessing some piece of shared information associated with the class. –  The syntax for a class method declaration is identical to that of an instance method, with one exception. •  Instead of using a minus sign for the method type identifier, you use a plus (+) sign. Based on “Learning Objective-C: A Primer”
  51. 51. Instances, methods, and messaging •  Class methods –  Example: •  In this case, the array method is a class method on the NSArray class - and inherited by NSMutableArray - that allocates and initializes a new instance of the class and returns it to your code. NSMutableArray *myArray = nil; // nil is essentially the same as NULL // Create a new array and assign it to the myArray variable. myArray = [NSMutableArray array]; Based on “Learning Objective-C: A Primer”
  52. 52. Properties •  Properties are a shortcut for creating getter/ setter methods •  Can still use dot notation because getter/setter methods are being used under the hood double speed = myCar.speed; •  Attributes of the property are declared with the property itself: – read-only/read-write access – Memory management policy
  53. 53. Properties (.h file) @interface Car : NSObject { double speed_; } @property (nonatomic, assign) double speed; @end
  54. 54. Properties (.m file) •  @synthesize - creates the getter/setter methods for you based on declaration •  @dynamic - allows you to create the getter/ setter methods
  55. 55. Properties (.m file) •  Example: @implementation Car @synthesize speed = speed_; //rest of your class implementation @end
  56. 56. read only/read-write @property double speed; // read-write by default @property (readonly) double speed;
  57. 57. Initializing Instances •  2 steps: –  Allocate –  Initialize •  Example: Car *myCar = [[Car alloc] init]; –  the myCar var is a pointer, but that s all you need to know about pointers •  Some classes have additional initialization methods Car *myCar = [[Car alloc] initWithPassengers:2];
  58. 58. Types •  id - a reference to any object –  Car *myCar; (static typing) –  id myCar; (notice no *)(dynamic typing) •  NSObject vs. id –  id is a placeholder for anything –  NSObject is the root class –  No compile-time warning when sending messages to id –  Messages sent to an NSObject will be checked
  59. 59. nil •  nil represents nothing or null if (myCar == nil) { do something } if (!myCar) { do something } Car *myCar = nil; •  Messages CAN be sent to nil: Car *myCar = nil; [myCar accelerate];
  60. 60. BOOL • Assigned values of YES or NO BOOL isCarOn = NO; if (isCarOn == YES) if (isCarOn) if (!isCarOn) isCarOn = 1;
  61. 61. The Class object •  An object that gives details about a given class (i.e. name, inheritance, etc.) Class *mustangClass = [Mustang class]; [mustang isSubclassOfClass:[Car class]]; •  Introspection - asking a class about itself if ([myObject isKindOfClass:[Car class]]) { // do something, since we know it’s a Car Car *myCar = (Car*)myObject; }
  62. 62. Identity Equality •  Identity - if two pointers point to the same memory if (objectA == objectB) { //objectA and objectB are the same instance } •  Equality - if contents are semantically equal if ([objectA isEqual:objectB]) { //different instances, but conceptually equal }
  63. 63. NSObject •  Root class •  Built-in implementation that all sub-classes inherit –  Memory management –  Introspection –  Object equality
  64. 64. NSObject’s description method •  NSObject defines the description method: - (NSString*)description; –  By default, returns memory location and type –  You can override this to show more meaningful info
  65. 65. NSString •  Abstracts unicode characters •  Always use this instead of char* (unless you absolutely need to) –  If you re invoking a C method that requires a char*, you can get char* from an NSString through the UTF8String method NSString *myString = @”Hello, World!”;
  66. 66. NSString (cont.) •  Formatting strings NSString *name = @”Bob” NSString *hello = [NSString stringWithFormat:@”Hello, %@”, name]; –  uses standard printf formatting •  Logging strings NSLog([myObject description]); NSLog(@”Hello, %@”, name); •  Lots of other useful methods on NSString (see documentation)
  67. 67. NSString (cont.) •  NSString is immutable •  @”hello” is an NSString –  ex, if a method exists on Logger class: - (void)logMessage:(NSString*)message; –  calling it with a string: Logger *logger = [[Logger alloc] init]; [logger logMessage:@”Hello, World!”]; –  No need to instantiate an NSString object, just pass the string in directly
  68. 68. Collections •  NSArray - an ordered collection of objects •  NSDictionary - collection of key-value pairs •  NSSet - unordered collection of unique objects –  These are all immutable collections. •  Mutable versions: NSMutableArray, NSMutableDictionary, NSMutableSet –  Careful with exposing these as properties
  69. 69. NSNumber •  A wrapper class that abstracts the concept of a number (int, double, float, etc). NSNumber *myInt = [NSNumber numberWithInt:10]; NSNumber *myDouble = [NSNumber numberWithDouble: 5.2]; int a = [myInt intValue]; double b = [myDouble doubleValue]; •  Useful when you need to store a number in a dictionary (because dictionaries only accept NSObjects).
  70. 70. Example •  Car
  71. 71. Memory management in ObjC •  Referencing Counting –  Every object has a retain count (how many other objects are interested in it) •  As long as retain count 0, object is alive •  When retain count drops to 0, memory is freed
  72. 72. Referencing Counting •  [myObject retain] will increment retain count •  [myObject release] will decrement retain count •  Once retain count = 0, object is no longer available, no turning back.
  73. 73. Referencing Counting •  Calling alloc sets retain count to 1 Car *myCar = [[Car alloc] init]; –  Retain count on myCar = 1
  74. 74. Dealloc •  The dealloc method deallocates memory for an object •  dealloc is never explicitly called by your code!!! NSObject handles this call for you. •  In your dealloc method, you release any objects you have a reference to
  75. 75. Dealloc You override dealloc when you need to clean up (release) your own iVars - (void)dealloc { //clean up any objects referenced [super dealloc]; } -  (void)dealloc { [name_ release]; //clean up any objects referenced [super dealloc]; }
  76. 76. Ownership You will typically use properties, which generate getter/ setter methods automatically. Internally, this is what they look like (or how you would code it if you did it yourself). - (void)setCar:(Car*)car { - (void)setName:(NSString*)name { [car_ autorelease]; [name_ autorelease]; car_ = [car retain]; name_ = [name copy]; } }
  77. 77. Autorelease •  Solves the problem where you need to return a newly created object before releasing it. •  Example: + (NSString*)fullNameWithFirstName:(NSString*)first lastName:(NSString*)last { NSString *full = [[NSString alloc] initWithFormat:@”%@ %@”, first, last]; [full autorelease]; return full; }
  78. 78. Autorelease •  Use the same release rules as before •  Even if retain count goes to 0, object will stick around for a while longer •  System will automatically get rid of it –  Q: What manages auto-released objects? –  A: Auto release pools
  79. 79. Autorelease Pools •  You typically don t create them yourself (except for threading cases) •  When an autorelease pool is drained,” any object that was sent the message “autorelease” will get released •  On the main thread, this is done automatically –  At the end of each run loop execution •  Avoid autoreleasing if possible. Better to alloc/ release (frees memory right away).
  80. 80. Rules Conventions •  When do you release an object? •  Methods that contain the words alloc or copy return a retained object - you must release it •  Any other method is returning an autoreleased object. •  You should follow this convention when defining your own methods!
  81. 81. Design Patterns •  A design pattern is usually defined as “a template for a design that solves a general, recurring problem in a particular context” •  Use them! (don’t go against them) •  Design patterns used often in iOS development: –  Model-View-Controller (MVC) –  Delegate –  Target-Action
  82. 82. MVC •  Model - The entity that models the data •  View - The user interface •  Controller - Manages the connection between view and model
  83. 83. MVC - Model •  The model should only be concerned with the data. •  Nothing to do with visual representation •  Can be persisted •  Models should be reusable
  84. 84. MVC - View •  The presentation of the data (or model) •  Allows user to interact with the data •  Does not store data, except for caching •  Easily reusable
  85. 85. MVC - Controller •  Brings the Model and View together •  Receives events from View, and acts on the Model •  Updates the View when the Model changes •  This is where app logic is typically written
  86. 86. Delegation •  Delegation is a design pattern in which one object in a program acts on behalf of, or in coordination with, another object. •  The delegating object keeps a reference to the other object (the delegate) and at the appropriate time sends a message to it. •  The message informs the delegate of an event that the delegating object is about to handle or has just handled. •  The delegate may respond to the message by updating the appearance or state of itself or other objects in the application, and in some cases it can return a value that affects how an impending event is handled.
  87. 87. Delegation •  The main value of delegation: it allows you to easily customize the behavior of several objects in one central object. •  This concept is closely related to the concept of a callback function, used, for example, in Ajax-based web development. –  A callback is a function that is triggered when an event occurs. –  Since we usually don’t know when the event will occur, we set a callback that acts as a listener for that event. –  In Cocoa Touch, callbacks are implemented using delegation. •  Most iOS applications will include delegates for some of their classes.
  88. 88. Delegation •  The concept of delegation is closely related to the Objective-C construct known as protocol. •  A protocol is simply a list of method declarations, unattached to a class definition. •  Protocols define an interface that other objects are responsible for implementing. •  When you implement the methods of a protocol in one of your classes, your class is said to conform to that protocol. •  Protocols are used to specify the interface for delegate objects.
  89. 89. Target-Action •  Allows you to handle actions or events by a UI control •  Some events: –  touchDown –  touchUpInside –  valueChanged
  90. 90. Target-Action •  3 things are defined: –  Target myObject –  Action @selector(convertFtoC:) –  Event UIControlEventTouchUpInside
  91. 91. Application event loop
  92. 92. MVC in Cocoa Touch •  Create your View components using Interface Builder –  Sometimes you ll create / modify your interface from code. –  You might also subclass existing views and controls. •  Model = Objective-C classes designed to hold your application s data or data model using Core Data •  Controller = typically composed of classes that you create and that are specific to your app. –  Controllers can be completely custom classes (NSObject subclasses), but more often, they will be subclasses of one of several existing generic controller classes from the UIKit framework, such as UIViewController. –  By subclassing one of these existing classes, you will get a lot of functionality for free and won t need to spend time recoding the wheel, so to speak.
  93. 93. Outlets •  Outlets –  Our controller class can refer to objects in the nib file by using a special kind of instance variable called an outlet. –  Think of an outlet as a pointer that points to an object within the nib. •  Example: suppose you created a text label in Interface Builder and wanted to change the label s text from within your code. By declaring an outlet and connecting that outlet to the label object, you could use the outlet from within your code to change the text displayed by the label.
  94. 94. Actions •  Actions –  Going in the opposite direction, interface objects in our nib file can be set up to trigger special methods in our controller class. –  These special methods are known as action methods. •  Example: you can tell Interface Builder that when the user touches up (pulls a finger off the screen) within a button, a specific action method within your code should be called.
  95. 95. Outlets •  More on Outlets –  Outlets are instance variables that are declared using the keyword IBOutlet. •  A declaration of an outlet in your controller s header file might look like this: @property (nonatomic, retain) IBOutlet UIButton *myButton;
  96. 96. Outlets •  More on Outlets –  IBOutletdoes absolutely nothing as far as the compiler is concerned. •  Its sole purpose is to act as a hint to tell Interface Builder that this is an instance variable that we re going to connect to an object in a nib file. •  Any instance variable that you create and want to connect to an object in a nib file must be preceded by the IBOutlet keyword
  97. 97. Outlets •  Recent changes –  Before iOS 4: IBOutlet UIButton *myButton; –  Since then, Apple s sample code has been moving toward placing the IBOutlet keyword in the property declaration: @property (nonatomic, retain) IBOutlet UIButton *myButton; –  Both mechanisms are supported.
  98. 98. Actions •  Actions are methods that are part of your controller class. –  They are also declared with a special keyword, IBAction, which tells Interface Builder that this method is an action and can be triggered by a control. –  Typically, the declaration for an action method will look like this: - (IBAction)doSomething:(id)sender; –  The actual name of the method can be anything you want, but it must have a return type of IBAction, which is the same as declaring a return type of void.
  99. 99. Actions •  If you don t need to know which control called your method, you can also define action methods without a sender parameter: - (IBAction)doSomething;
  100. 100. Example •  Button Fun –  Creating the View Controller •  ButtonFun_ViewController.h
  101. 101. Example ButtonFun_ViewController.m
  102. 102. Example ButtonFun_ViewController.m
  103. 103. Example •  Button Fun –  Creating the App Delegate Button_FunAppDelegate.h
  104. 104. Example •  Button Fun –  The UIApplicationDelegate protocol
  105. 105. Example Button_FunAppDelegate.m
  106. 106. Example Button_FunAppDelegate.m
  107. 107. Example •  Button Fun –  Editing MainWindow.xib
  108. 108. Example •  Button Fun –  Editing ButtonFun_ViewController.xib
  109. 109. Example •  Button Fun –  Screenshots
  110. 110. Another Example •  SayMyName –  Screenshots
  111. 111. UI design aspects •  The iOS development environment adheres to human interface guidelines, strategies and best practices. –  See http://tinyurl.com/3yj7b5y •  Each iPhone or iPad app consists of familiar UI elements, organized in a somewhat predictable way, and associated with equally predictable actions. •  UI elements can be selected, configured, and customized interactively using Interface Builder. –  They can also be created and manipulated programmatically.
  112. 112. A guided tour of UI elements
  113. 113. Example •  Control Fun
  114. 114. Camera, photo library, image manipulation •  The iPhone, iPod touch, and second-generation iPad each have one or two built-in cameras and a built-in application called Photos that allows you to manage the device’s photo and video library. •  Due to the “sandboxed” nature of iOS, an application cannot have direct access to photographs or any other data that live outside of their own sandboxes. –  This limitation has been circumvented by exposing the camera and the media library to other applications by way of an image picker mechanism.
  115. 115. Camera, photo library, image manipulation •  The main classes that you need to understand in order to develop basic applications involving images, camera, and photo library for the iPhone are: –  UIImageView –  UIImagePickerController •  (and its associated protocol, UIImagePickerControllerDelegate).
  116. 116. UIImageView •  An image view object provides a view-based container for displaying either a single image or for animating a series of images. •  New image view objects are configured to disregard user events by default. –  If you want to handle events in a custom subclass of UIImageView, you must explicitly change the value of the userInteractionEnabled property to YES after initializing the object. •  When a UIImageView object displays one of its images, the actual behavior is based on the properties of the image and the view. –  If either of the image’s leftCapWidth or topCapHeight properties are non-zero, then the image is stretched according to the values in those properties. –  Otherwise, the image is scaled, sized to fit, or positioned in the image view according to the contentMode property of the view.
  117. 117. UIImageView •  Apple recommends that you use images that are all the same size. –  If the images are different sizes, each will be adjusted to fit separately based on that mode. •  All images associated with a UIImageView object should use the same scale. –  If your application uses images with different scales, they may render incorrectly.
  118. 118. UIImagePickerController •  The UIImagePickerController class provides basic, customizable user interfaces for taking pictures and movies and for giving the user some simple editing capability for newly-captured media. •  The role and appearance of an image picker controller depend on the source type you assign to it before you present it: –  A sourceType of UIImagePickerControllerSourceTypeCamera provides a user interface for taking a new picture or movie (on devices that support media capture). –  A sourceType of UIImagePickerControllerSourceTypePhotoLibrary or UIImagePickerControllerSourceTypeSavedPhotosAlbum provides a user interface for choosing among saved pictures and movies.
  119. 119. UIImagePickerControllerDelegate•  The UIImagePickerControllerDelegate protocol defines methods that your delegate object must implement to interact with the image picker interface. •  The methods of this protocol notify your delegate when the user either picks an image or movie, or cancels the picker operation. •  The delegate methods are responsible for dismissing the picker when the operation completes.
  120. 120. The UIImage class •  A UIImage object is a high-level way to display image data. •  Images can be created from files, from Quartz image objects, or from raw image data received by the application. •  The UIImage class also offers several options for drawing images to the current graphics context using different blend modes and opacity values. •  The UIImage class supports the most popular image file formats.
  121. 121. The UIImage class •  Recommendations and warnings: –  Image objects are immutable, i.e., you cannot change their properties after creation. •  This means that you generally specify an image’s properties at initialization time or rely on the image’s metadata to provide the property value. •  Because image objects are immutable, they also do not provide direct access to their underlying image data. –  In low-memory situations, image data may be purged from a UIImage object to free up memory on the system. •  This purging behavior affects only the image data stored internally by the UIImage object and not the object itself. –  Avoid creating UIImage objects that are greater than 1024 × 1024 in size.
  122. 122. Example •  Camera –  Illustrates the image picker mechanism
  123. 123. Example •  Camera
  124. 124. The Media Layer •  The Media Layer of the iOS architecture contains the graphics, audio, and video technologies that provide multimedia support on a mobile device. •  The simplest (and most efficient) way to create an application is to use pre-rendered images together with the standard views and controls of the UIKit framework and let the system do the drawing. –  However, there may be situations where additional technologies are needed to manage the application’s graphical content.
  125. 125. The Media Layer •  Core Graphics framework: contains the interfaces for the Quartz 2D drawing API, which handles native 2D vector- and image-based rendering. •  Core Animation (part of the Quartz Core framework): provides advanced support for animating views and other content. •  OpenGL ES framework: provides support for 2D and 3D rendering using hardware-accelerated interfaces.
  126. 126. The Media Layer •  Core Text framework: contains a set of simple, high- performance C-based interfaces for laying out text and handling fonts. •  Image I/O: provides interfaces for reading and writing most image formats and associated image metadata. –  This framework makes use of the Core Graphics data types and functions and supports all of the standard image types available in iOS. •  Assets Library framework: provides a query-based interface for retrieving photos and videos from the user’s device.
  127. 127. Quartz 2D •  Advanced, two-dimensional drawing engine available for iOS application development. •  Provides low-level, lightweight 2D rendering with unmatched output fidelity regardless of display or printing device. •  Resolution- and device-independent. •  The Quartz 2D application programming interface (API) is easy to use and provides access to powerful features. –  The Quartz 2D API is part of the Core Graphics framework, so you may see Quartz referred to as Core Graphics or, simply, CG. •  In iOS, Quartz 2D works with all available graphics and animation technologies, such as Core Animation, OpenGL ES, and the UIKit classes.
  128. 128. Quartz 2D •  The painter’s model
  129. 129. Quartz 2D •  Graphics context –  A graphics context represents a drawing destination. •  It contains drawing parameters and all device-specific information that the drawing system needs to perform any subsequent drawing commands. –  A graphics context defines basic drawing attributes such as the colors to use when drawing, the clipping area, line width and style information, font information, compositing options, etc. –  A graphics context is represented in code by the data type CGContextRef. •  After you obtain a graphics context, you can use Quartz 2D functions to draw to the context, perform operations (such as translations) on the context, and change graphics state parameters, such as line width and fill color.
  130. 130. Quartz 2D •  Bitmap graphics context –  Allows you to paint RGB colors, CMYK colors, or grayscale into a bitmap. •  A bitmap is a rectangular array (or raster) of pixels, each pixel representing a point in an image. •  Bitmap images are also called sampled images. –  A bitmap graphics context accepts a pointer to a memory buffer that contains storage space for the bitmap. •  When you paint into the bitmap graphics context, the buffer is updated. •  After you release the graphics context, you have a fully updated bitmap in the pixel format you specify.
  131. 131. Creating a bitmap graphics context
  132. 132. Drawing to a bitmap graphics context
  133. 133. Bitmap images and image masks •  A bitmap image is an array of pixels, where each pixel represents a single point in the image. •  Bitmap images are restricted to rectangular shapes, but with the use of the alpha component, they can appear to take on a variety of shapes and can be rotated and clipped. •  Each sample in a bitmap contains one or more color components in a specified color space, plus one additional component that specifies the alpha value to indicate transparency. –  Each component can be from 1 to as many as 32 bits.
  134. 134. Bitmap images and image masks •  When you create and work with Quartz images (which use the CGImageRef data type), you will notice that some Quartz image-creation functions require that you specify all this information, while other functions require a subset of this information. –  What you provide depends on the encoding used for the bitmap data, and whether the bitmap represents an image or an image mask.
  135. 135. Functions for creating images
  136. 136. Image masks •  An image mask is a bitmap that specifies an area to paint, but not the color. •  A Quartz bitmap image mask is used the same way an artist uses a silkscreen. •  A bitmap image mask determines how color is transferred, not which colors are used. •  Each sample value in the image mask specifies the amount that the current fill color is masked at a specific location. •  The sample value specifies the opacity of the mask. –  Larger values represent greater opacity and specify locations where Quartz paints less color.
  137. 137. Image masks •  The function CGImageMaskCreate creates a Quartz image mask from bitmap image information that you supply.
  138. 138. Image I/O •  The Image I/O programming interface allows applications to read and write most image file formats. –  Originally part of the Core Graphics framework, Image I/O now resides in its own framework to allow developers to use it independently of Core Graphics (Quartz 2D). •  Image I/O provides the definitive way to access image data because it is highly efficient, allows easy access to metadata, and provides color management. •  The Image I/O framework provides opaque data types for reading image data from a source (CGImageSourceRef) and writing image data to a destination (CGImageDestinationRef). •  The Image I/O framework understands most of the common image file formats.
  139. 139. iPad •  Different considerations must be made when building an iPad app •  Because of the larger screen, users expect a more immersive experience •  You do NOT want to simply auto-size your iPhone app to stretch out to the new dimensions –  This is a horrible waste of space, and makes for a poor UX
  140. 140. iPad •  Some new controls –  UISplitViewController –  Popovers •  Other differences from iPhone –  Due to the physical design of the iPad, it is much easier to turn the device to a different orientation. Therefore, it is assumed that iPad apps support all orientations •  This is not the case with iPhone
  141. 141. UISplitViewController
  142. 142. Popovers
  143. 143. iPad •  Two ways to build for iPad: –  Create an iPad-only project •  All code will be specific for iPad •  It is more difficult to reuse code this way •  Good if you only plan on releasing an iPad app –  Create a universal app •  Will support both iPhone and iPad in one download •  Allows you to easily share code
  144. 144. Building an iPad-only project •  Use the same concepts as you would with an iPhone app –  MVC –  Delegation –  Etc. •  The only real difference is how views are laid out on the screen (be sure to use the screen real estate appropriately).
  145. 145. Building a universal app •  Still use the same patterns from iPhone development •  The project will consist of 2 different versions of each of your views –  If your views are built in NIB files, then you would have 2 NIB files for each view: one for iPhone, and another for iPad –  If you are subclassing UIView for any of your views, you will have 2 subclasses: one for iPhone, and another for iPad
  146. 146. Building a universal app •  Your project will consist of 2 versions of each of your view controllers –  The problem here is that the view controllers typically contain business logic (business rules, logical flow of the app, etc.) –  You do not want to duplicate business logic –  The answer: •  Create a base class for each view controller •  This base class contains shared code (anything that is reusable between iPhone/iPad) •  You then create 2 subclasses of this base class: one for iPhone, and another for iPad •  These subclasses contain the details that only pertain to that form.
  147. 147. Building a universal app •  Creating subclasses is fine, but how does the app know which one to create at runtime? –  It all starts from the main NIB file –  In a universal app, there are 2 main NIB files: one for iPhone, and another for iPad –  The PLIST configuration file settings indicate which NIB gets loaded for a specific form factor –  Your job: Customize these NIB files so that it starts loading the proper classes –  This creates a chain reaction
  148. 148. Building a universal app •  What about just having an IF statement that checks for the type of device? if  (device  ==  iPad)    do  something  for  iPad   else    do  something  for  iPhone  •  NO!!! This is bad! –  Conceptually this may work (and it would), however the logic in your view controllers becomes cluttered with IF-ELSE statements –  Very difficult to maintain •  Apple’s recommended method: create subclasses instead of IF-ELSE logic
  149. 149. Part II OpenCV and iOS
  150. 150. OpenCV •  OpenCV (Open Source Computer Vision) is a library of programming functions for real- time computer vision. •  OpenCV is released under a BSD license; it is free for both academic and commercial use. •  Goal: to provide a simple-to-use computer vision infrastructure that helps people build fairly sophisticated vision applications quickly. •  The library has 2000+ optimized algorithms. –  It is used around the world, has 2M downloads and 40K people in the user group.
  151. 151. OpenCV •  5 main components: 1.  CV: basic image processing and higher-level computer vision algorithms 2.  ML: machine learning algorithms 3.  HighGUI: I/O routines and functions for storing and loading video and images 4.  CXCore: basic data structures and content upon which the three components above rely 5.  CvAux: defunct areas + experimental algorithms; not well-documented.
  152. 152. OpenCV •  Image representation: IplImage –  Basic structure used to encode [grayscale, color, or four-channel (RGB + alpha)] images. •  Moreover, each channel may contain any of several types of integer or floating-point numbers, which makes for a very flexible representation. •  For practical purposes, we can say that an IplImage is derived from another class, CvMat, which can be though of as inherited from a third, abstract base class, CvArr.
  153. 153. OpenCV •  Image processing and computer vision •  Machine learning –  Filters –  K-means clustering algorithm –  Morphological operators –  Naive Bayes classifier –  Geometric transforms –  Binary decision trees –  Edge detection –  Boosting algorithms, e.g., AdaBoost –  Hough transform –  Viola-Jones classifier and its application –  DFT, DCT to face detection –  Histogram-based matching –  Expectation maximization (EM) –  Template matching clustering –  Freeman codes –  K-nearest neighbors (KNN) classifier –  Shape matching –  Support vector machine (SVM) –  Foreground-background segmentation –  etc. –  Inpainting –  Corner and interest point detection –  Optical flow –  Object tracking in video sequences –  Extensive support for stereo imaging –  etc.
  154. 154. OpenCV and iOS •  Since OpenCV is not available as a bundled library in Apple’s iOS framework, we must compile and bring in the OpenCV libraries and headers ourselves. •  Step-by-step setup described in detail at http://niw.at/articles/2009/03/14/using-opencv-on-iphone/en –  Follow the instructions and look for possible updates. •  Once you have completed the setup process, you should have access to both compiled versions of OpenCV: –  one used when building our applications to test on the iOS Simulator; –  one needed for final deployment to the actual device.
  155. 155. OpenCV and iOS •  When working with OpenCV in the iOS SDK: 1.  Convert the UIImage object to an IplImage structure; 2.  Apply OpenCV library function(s); 3.  Convert IplImage back to a UIImage before attempting to display it on the screen.
  156. 156. OpenCV and iOS •  Code to convert from UIImage to IplImage –  Note that IplImage also needs to be released by using cvReleaseImage.
  157. 157. OpenCV and iOS •  Code to convert from IplImage to UIImage
  158. 158. OpenCV and iOS •  Edge detection code snippet
  159. 159. OpenCV and iOS •  Example: OpenCV sample app –  Based on code from Yoshimasa Niwa –  Demonstrates two functions: •  Edge detection •  Face detection
  160. 160. OpenCV and iOS •  Main methods –  opencvFaceDetect: iOS-compatible wrapper around the OpenCV library function that implements the Haar classifier (cvHaarDetectObjects). –  opencvEdgeDetect: iOS-compatible wrapper around the OpenCV library function that implements the Canny edge detector (cvCanny).
  161. 161. OpenCV and iOS DEMO
  162. 162. Part III Examples and case studies
  163. 163. OpenGL ES •  OpenGL is “the most widely adopted graphics standard in the computer graphics and gaming industry”. •  The OpenGL ES framework (OpenGLES.framework) provides tools for drawing 2D and 3D content. •  It’s a C-based framework that works closely with the device hardware to provide high frame rates for full-screen game-style applications.
  164. 164. OpenGL ES •  Example: GLImageProcessing sample app –  Demonstrates how to implement simple image processing filters (Brightness, Contrast, Saturation, Hue rotation, Sharpness) using OpenGL ES1.1. –  Also shows how to create simple procedural button icons using CoreGraphics. –  By looking at the code youll see how to set up an OpenGL ES view and use it for applying a filter to a texture. •  The application creates a texture from an image loaded from disk. –  Use the slider to control the current filter. •  Only a single filter is applied at a time. •  Result cannot be saved.
  165. 165. OpenGL ES •  Screenshots
  166. 166. OpenGL ES DEMO
  167. 167. Image tagging using the IQEngines API •  IQEngines is the brain behind the oMoby iPhone app)
  168. 168. Image tagging using the IQEngines API
  169. 169. The IQEngines API •  Query API: allows the user to send an image to IQ Engines’ server to be processed by IQ Engines’ image labeling engine. –  It expects an HTTP callback as one of its arguments; as soon as the images are labeled, the results will be posted to that URL. •  Update API: a long polling request that returns the results for one or more images as soon as they have been labeled. •  Training API: allows you to upload images to the IQ Engines’ object search database (alpha trials, by request only). •  Result API: returns the labels for a specific image if it has been successfully processed, or informs that the image is still being processed. •  Crowdsource API: “coming soon”.
  170. 170. The IQEngines API demo app •  Screenshots
  171. 171. The IQEngines API demo app •  XML-formatted response
  172. 172. The IQEngines API demo app DEMO
  173. 173. Other third-party APIs •  Successful photo-based iPhone app developers are starting to release APIs for third-party developers. •  Recent examples: –  PicPlz has made their API (+ examples) available earlier this year –  Instagram appears to have followed suit a few days later
  174. 174. iTranslatAR •  By Julie Carmigniani [Florida Atlantic University] •  An app for translating pictures of text into any language supported by Google Translate. •  The picture’s text is converted into a string using the Tesseract OCR software (now available at Google Code)
  175. 175. iTranslatAR •  The app currently has three views in a tab controller: –  a view for translating text through pictures; –  one for translating text through typing; and –  one for changing the target language.
  176. 176. iTranslatAR •  Screenshots
  177. 177. iTranslatAR DEMO
  178. 178. iTranslatAR •  Will be extended to include (near) real-time camera support inspired by Word Lens and augmented reality (the AR in iTranslatAR) capabilities.
  179. 179. An integrated app: Photastic •  By Asif Rahman [Florida Atlantic University]
  180. 180. An integrated app: Photastic DEMO
  181. 181. Concluding thoughts •  Mobile image processing, image search, and computer vision-based apps have a promising future. •  There is a great need for good solutions to specific problems. •  I hope this mini-course has provided a good starting point and many useful pointers. •  I look forward to working with some of you!
  182. 182. Thank You •  Questions? •  For additional information: omarques@fau.edu