How to Recognize Rotated Text with OCR and Barcode SDK

The accuracy of optical character recognition (OCR) is sensitive to text orientation. If a text image file is rotated by 90°, 180° or 270°, and there is no EXIF information, it would be challenging to doing text recognition. In case of text around barcode, the combination of Dynamsoft Label Recognition(DLR) and Dynamsoft Barcode Reader(DBR) is proposed as an ideal OCR solution. This article will demonstrate how to build a C++ command-line application to recognize a rotated barcode label (e.g. price tag) which is hardly recognized by general open source or commercial OCR SDK.

Requirements

DLR API Documentation

https://www.dynamsoft.com/label-recognition/programming/c-cplusplus/user-guide.html?ver=latest

A New OCR Approach: Text Recognition by Barcode Orientation Detection

Both Dynamsoft Barcode SDK and Dynamsoft OCR SDK support JSON-formatted template import. To parse JSON in C++, we use json.hpp, which is a single header file for easy integration. No dependencies required.

A basic template file for DLR runtime settings is as follows:

{
"CharacterModelArray" : [
{
"Name" : "NumberLetter"
}
],
"LabelRecognitionParameterArray" : [
{
"Name" : "BarcodeLabel",
"ReferenceRegionNameArray" : [ "defaultReferenceRegion" ]
}
],
"ReferenceRegionArray" : [
{
"Localization" : {
"BarcodeFormatIds" : [ "DLR_BF_ALL" ]
},
"Name" : "defaultReferenceRegion"
}
]

}

The BarcodeFormatIds is mandatory. If the input barcode symbologies are not fixed, we set DLR_BF_ALL for covering primary 1D and 2D barcode symbologies.

Let’s create DLR and DBR objects and construct a JSON object by loading the template file:

#include <string>
#include <fstream>
#include <iostream>

#include "include/DynamsoftLabelRecognition.h"
#include "include/DynamsoftCommon.h"
#include "include/DynamsoftBarcodeReader.h"
#include "include/json.hpp"

using json = nlohmann::json;
using namespace dynamsoft::dlr;
using namespace dynamsoft::dbr;

CLabelRecognition dlr;
dlr.InitLicense("LICENSE-KEY");

Before invoking the OCR APIs, we determine the reference regions based on barcode recognition results:

TextResultArray *resultArray = NULL;
dbr.DecodeBuffer(imageCopy.data, imgWidth, imgHeight, imageCopy.step.p[0], IPF_RGB_888,"");
dbr.GetAllTextResults(&resultArray);

dlr.ResetRuntimeSettings();
DLRRuntimeSettings settings = {};
dlr.GetRuntimeSettings(&settings);
string templateName = templateObj["LabelRecognitionParameterArray"][0]["Name"];
dlr.AppendSettingsFromString(templateObj.dump().c_str());
dlr.UpdateReferenceRegionFromBarcodeResults(resultArray, templateName.c_str());
dlr.UpdateRuntimeSettings(&settings);

Here are the decoding APIs used for different data source types:

Afterward, we can either use RecognizeByBuffer or RecognizeByFile to recognize text:

// buffer
DLRImageData data = {imageCopy.step.p[0] * imgHeight, imageCopy.data, imgWidth, imgHeight, imageCopy.step.p[0], DLR_IPF_RGB_888};
dlr.RecognizeByBuffer(&data, templateName.c_str());

// file
dlr.RecognizeByFile("<image file>", templateName.c_str());

The returned result is an array, from which we can get text and corresponding coordinate values:

DLRResultArray* pDLRResults = NULL;
dlr.GetAllDLRResults(&pDLRResults);
if (pDLRResults != NULL)
{
int rCount = pDLRResults->resultsCount;
for (int ri = 0; ri < rCount; ++ri)
{
DLRResult* result = pDLRResults->results[ri];
int lCount = result->lineResultsCount;

for (int li = 0; li < lCount; ++li)
{
printf("Line result %d: %s\r\n", li, result->lineResults[li]->text);
DLRPoint *points = result->lineResults[li]->location.points;
printf("x1: %d, y1: %d, x2: %d, y2: %d, x3: %d, y3: %d, x4: %d, y4: %d\r\n", points[0].x,
points[0].y, points[1].x, points[1].y, points[2].x, points[2].y, points[3].x, points[3].y);
}
}
}
dlr.FreeDLRResults(&pDLRResults);
if (resultArray) CBarcodeReader::FreeTextResults(&resultArray);

Label OCR Test by Orientation

To prove the efficiency of combining Dynamsoft barcode and OCR SDKs, we can make a simple experiment by rotating a barcode label image.

Sample Image

0 degree

90 degree

180 degree

270 degree

As long as there is a barcode exists, the text aside can be successfully recognized.

Source Code

https://github.com/Dynamsoft/OCR-barcode-label

Originally published at https://www.dynamsoft.com on March 22, 2021.

Manager of Dynamsoft Open Source Projects | Tech Lover

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store