OCR in iOS 15, i.e. automatic detection of text in photos - how does it work?

1. SPIDER'S WEB
2. Technologies
3. Software

How does OCR work in iOS 15?

The mechanisms for optical image recognition have appeared in the gallery, among others, thanks to which we can recognize text on all photos and graphics stored in the iCloud Photos Library. Just go to the gallery, and if the algorithms detect some text on the selected image in the Photos application, a characteristic icon will appear in the lower right corner.

Importantly, the photo does not have to be taken with an iPhone for the OCR mode to work. I didn't expect it to be possible, but I was able to select and copy the text from the bicycle frame, which I captured on a blurry 640 x 480 pixel photo imported to iCloud Photos Library… over 18 years ago.

Optical character recognition can also be started directly in the iPhone camera.

It is also a very useful solution, as it allows you to quickly scan, for example, a telephone number on an advertisement or menu in a restaurant. The content is recognized even before the photo is taken and you can interact with it - although you should be careful not to wave your hand too much, and the signs do not fall out of the frame when you try to bump them.

Interestingly, the iPhone camera with OCR mode turned on can be started ... in any text field. After bumping into them, a new icon appears next to the "paste" button, the same as in the gallery, which starts the preview from the lens and automatically pastes letters that it recognizes - and this particular implementation of OCR just works so far ... so-so.

OCR works great on just taken screenshots.

Websites (usually) allow you to copy content from them without a problem, but in applications this is often impossible - and this is where this new OCR feature comes in handy for use in a screenshot. The content obtained in this way can then be copied to the clipboard in iOS 15 and pasted to any other program, and I already know that this solution will be useful at work.

In addition, copying recognized letters, numbers and special characters to the clipboard and transferring them to other applications is only one of the uses of OCR in iOS 15. Words selected in this way can be immediately checked in a dictionary or translated, and you can call a phone number. I will certainly use this solution, especially when traveling abroad.

There is one problem with OCR in iOS15 - the company from California, as usual, respects Poles.

It's good that optical character recognition works in our region and we can use it to recognize e.g. telephone numbers, but unfortunately it does not support all Polish letters. This means that we will not always be able to check the scanned text in the dictionary, and when copying and pasting it, you always have to make sure that, for example, "ł" has not changed into "t".

I would like to say that there is hope that one of the iOS updates will add support for Polish characters, but I do not have hope. Years pass, and the list of iOS functions not available in Polish is growing. OCR joins the bar with word suggestions, typing by scribbling, the option to set two languages ​​at once, the keyboard on Apple Watch, the Translator application and Siri ...

PS The new OCR function in iOS 15 requires an A12 Bionic processor or newer, which means that the function can be run on iPhones from XR / XS up. For the novelty to appear, you also need to add, due to the lack of Polish language support, e.g. English as an additional language in the settings, and then select the checkbox next to the Live Text field.

iOS 15 and iPadOS 15 will hit Apple's smartphones and tablets on September 20, 2021. Currently, the company's customers can download the Release Candidate version of the system.

Tags: AppleiosiOS 15iPadipadosiPadOS 15iphoneOCR