/ 
Image-DecodeQR-WeChat-2.2
River stage zero No dependents
/ README.md

NAME

Image::DecodeQR::WeChat - Decode QR code(s) from images using the OpenCV/WeChat library via XS

VERSION

Version 2.2

SYNOPSIS

This module detects and decodes QR code(s) in an input image.

It provides a Perl interface to the C++ OpenCV/WeChat QR code decoder via XS code. OpenCV/WeChat library uses CNN (Convolutional Neural Networks) to do this with pre-trained models. It works quite well.

This module has been tested with OpenCV v4.5.5, v4.8.1, v4.9 and Perl v5.32, v5.38 on Linux. But check the CPANtesters matrix on the left for all the tests done on this module (although tests may be sparse and rare because of the OpenCV dependency).

The OpenCV/WeChat library is relatively successful even for non-orthogonally rotated codes. It remains to be tested fully on the minimum size of the code images. 60px was the minimum size with my tests. See section "TESTING THE QR CODE DETECTION ALGORITHM" for more details.

Here is some code to get you started:

use Image::DecodeQR::WeChat;

# decode QR image with convenient "named" params
my $ret = Image::DecodeQR::WeChat::detect_and_decode_qr({
    # these are required
    'input' => 'input.jpg',
    # optional with defaults:
    # specify a different models dir
    #'modelsdir' => '...',
    # dump results to file(s) whose name
    # is prepended by param 'outbase'
    #'outbase' => 'outs',
    # set this to 1 to also dump images of
    # QR codes detected as well text files with
    # payload and bounding box coordinates
    #'dumpqrimagestofile' => 0,
    # use OpenCV's GUI image viewer to display
    # this needs a non-headless OS and OpenCV highgui
    #'graphicaldisplayresult'' => 0,
    #'verbosity' => 0,
});
die "failed" unless $ret;

# we got back an array-of-2-arrays
# * one contains the QR-code-text (called payload)
# * one contains bounding boxes, one for each payload
# we have as many payloads and bounding boxes as
# are the QR-codes detected (some may have been skipped)

# the number of QR code images found in the input
# NOTE: the returned array will contain
# just 2 empty arrays if no QR code was detected
# so this is the right way:
my $num_qr_codes_detected = scalar @$payloads;

my ($payloads, $boundingboxes) = @$ret;
for (0..$#$payloads){
  print "Payload got: '".$payloads->[$_]
    ."' bbox: @{$boundingboxes->[$_]}"
    .".\n";
}

# Alternatively, a less convenient method to
# decode a QR code is via the XS sub.
# It requires that all parameters be specified
# unlike detect_and_decode_qr() which uses
# "named" parameters  with defaults.
my $ret = Image::DecodeQR::WeChat::detect_and_decode_qr_xs(
    # the input image containing one or more QR-codes
    'an-input-image.png',

    # the dir with model parameters required by the library.
    # Model files come with this Perl module and are curtesy of WeChat
    # which is part of OpenCV contrib packages.
    # They are installed with this module and their default location
    # is given by Image::DecodeQR::WeChat::modelsdir()
    # Alternatively, you specify here your own model files:
    Image::DecodeQR::WeChat::modelsdir(),

    # outbase for all output files, optional = set to undef,
    # if more than one QR-codes were detected then an index will
    # be appended to the filename. And there will be JPEG image files
    # containing the portion of the image which was detected
    # and there will be txt files with QR-code text (payload)
    # and its bounding box. And there will be an overall
    # text file with all payloads. This last one will be
    # printed to STDOUT if no outbase was specified:
    'output.detected',

    # verbosity level. 0:mute, 1:C code messages, 10:C+XS code
    10,

    # display results in a window with QR codes found highlighted
    # make sure you have an interactive shell and GUI
    1,

    # dump image and metadata to files for each QR code detected
    # only if outbase was specified
    1,
);
die "failed" unless $ret;

# again, the same data structure returned:
my ($payloads, $boundingboxes) = @$ret;
my $num_qr_codes_detected = scalar @$payloads;
for (0..$#$payloads){
  print "Payload got: '".$payloads->[$_]
    ."' bbox: @{$boundingboxes->[$_]}"
    .".\n";
}

# Where is it looking for default
# pre-trained models location ?
# (they are installed with this module)
print "my models are in here: ".Image::DecodeQR::WeChat::modelsdir()."\n"

# returns 1 or 0 when OpenCV was compiled with highgui or not
# and supports GUI display like imshow() which displays an image in a window
my $has_highgui_support = opencv_has_highgui_xs();

This code interfaces functions and methods from OpenCV/WeChat library (written in C++) for decoding one or more QR code images found embedded in images. It's just that: a very thin wrapper of a C++ library written in XS. It only interfaces the OpenCV/WeChat library for QR code decoding and accommodates its returned data into a Perl array (ref).

It can detect multiple QR codes embeded in a single image. It has been successfully tested with images as small as 60 x 60 pixels.

The payload (i.e. the QR-code's text) and the coordinates of the bounding box around each QR code image detected are returned back as Perl array of tuples (i.e. [[QR code text, bounding box]...).

Optionally, it can output the portion of the input image corresponding to each QR-code (that is a sub-image, part of the input image), its bounding box and the payload in separate files, useful for debugging and identification when multiple QR codes exist in a single input image.

FUTURE WORK

Following the XS code in this module as a guide, it will be trivial to interface other parts of the OpenCV library:

Ιδού πεδίον δόξης λαμπρόν
   (behold a glorious field of glory)

EXPORT

SUBROUTINES/METHODS

detect_and_decode_qr(\%params)

It tries to detect all the QR codes in the input image (specified by its filepath) with one or more QR codes embedded in it.

It wraps the XS function detect_and_decode_qr_xs(\@params) and replaces missing optional parameters with defaults.

These are the \%params it accepts:

Returned data by detect_and_decode_qr()

On success, it returns results back as an ARRAYref of 2-item-arrays (tuples) containing:

It returns an array of two zero-length arrays if no QR codes were detected.

It returns undef on failure.

Noting all the above, here is a way of calling it, checking its success and iterating over all the QR codes data returned:

my $ret = detect_and_decode_qr(\%params);
die "failed to detect_and_decode_qr()" unless defined $ret;
my ($payloads, $bounding_boxes) = @$ret;
# the number of detected QR codes (can be zero!):
my $num_qr_codes_detected = scalar @$payloads;
for ( 0 .. ($num_qr_codes_detected-1) ){
  print "payload: ".$payloads->[$_]."\n";
  print "bbox: ".$bounding_boxes->[$_]."\n";
}

detect_and_decode_qr_xs(infile, modelsdir, outbase, verbosity, graphicaldisplayresult, dumpqrimagestofile)

It tries to detect all the QR codes in the input image (specified by its filepath) with one or more QR codes embedded in it.

This is an XS function (which can be called safely by a Perl script) wrapped by detect_and_decode_qr(\%params) which is more convenient as it replaces missing parameters with defaults, unlike this function which expects all the parameters to be specified.

These are the @params it accepts, in this order:

It returns exactly the same results as detect_and_decode_qr(), see "Returned data by detect_and_decode_qr()" for details.

modelsdir()

It returns the path to the default location of the directory containing the pre-trained CNN models.

opencv_has_highgui_xs()

It returns 0 or 1 depending whether OpenCV's highgui library component was detected during this module's installation. If the result is 1 then the component is installed in your system and the graphicaldisplayresult parameter to both detect_and_decode_qr(\%params) and detect_and_decode_qr_xs(@params) can be set to 1. See CAVEATS for the efficacy of this subroutine.

COMMAND LINE SCRIPT

image-decodeqr-wechat.pl --input image-with-qr-code.jpg

image-decodeqr-wechat.pl --help

A CLI script is provided and will be installed by this module. Basic usage is as above. Here is its usage:

Usage : script/image-decodeqr-wechat.pl <options>

where options are:

  --input F :
    the filename of the input image
    which supposedly contains QR codes to be detected.

  --modelsdir M :
    optionally use your own models contained
    in this directory instead of the ones
    this program was shipped with.

  --outbase O :
    basename for all output files
    (if any, depending on whether --dumpqrimagestofile is on).

  --verbosity L :
    verbosity level, 0:mute, 1:C code, 10:C+XS code.

  --graphicaldisplayresult :
    display a graphical window with input image
    and QR codes outlined. Using --dumpqrimagestofile
    and specifying --outbase, images and payloads and
    bounding boxes will be saved to files, if you do
    not have graphical interface.

  --dumpqrimagestofile :
    it has effect only of --outbase was specified. Payloads,
    Bounding Boxes and images of each QR-code detected will
    be saved in separate files.

PREREQUISITES

INSTALLING OpenCV

In my case installing OpenCV using Linux's package manager (dnf, fedora) was not successful with default repositories. It required to add another repository (rpmfusion) which wanted to install its own versions of packages I already had. So I prefered to install OpenCV from sources. This is the procedure I followed:

Your mileage may vary.

If you are seriously in need of installing this module then consider migrating to a serious operating system such as Linux as your first action.

INSTALLING THIS MODULE

This module depends on the existence of the OpenCV library with all the extensions and contributed modules mentioned in section "PREREQUISITES".

Detecting where this library is located in your system is the weakest link in the installation process of this module. Makefile.PL contains code to do this with pkg-config or cmake. If these fail, it will look for ENVironment variables: OPENCV_LDFLAGS and OPENCV_CFLAGS, which should contain the CFLAGS (for example: -I/usr/include/opencv4/) and LDFLAGS (for example: -L/usr/lib64 -lopencv_world). Set these variables manually prior installation if the automatic methods mentioned above fail.

One last thing to check is that if your OpenCV installation (developer version) was correct, there should be a pkg-config file, perhaps in /usr/lib64/pkgconfig/opencv4.pc or /usr/local/lib64/pkgconfig/opencv4.pc. This file details all the CFLAGS and LDFLAGS and should be found by Makefile.PL if it is in a standard location, or adjust the list of paths in environment variable PKG_CONFIG_PATH which is where pkg-config searches for these files.

TESTING THE QR CODE DETECTION ALGORITHM

You will want to produce QR codes in order to assess how well the algorithm (basically the CNN pre-trained models supplied by OpenCV/WeChat) detects codes and find out the minimum size, quality, rotation angles etc. for optimal detection.

In producing test images from an original QR code, with software like the GIMP, one should be aware of the distortions caused by transforms such as scale and rotation to the final QR code images, rotation in particular. Add certain enhancements to the final image to "look good" and the resultant image looks like QR code but it is not. Failure of the library on such artificially produced images would be somehow expected.

Instead I would suggest testing with images which have been scanned with a QR code image attached to them in random angles and zoom factors. Using a photocopier or a scanner.

My use case was to process scanned images with a glued-in QR code tag which is detected and archive the document from the scanner to the appropriate files.

UNIT TESTING

There are two sets of tests which can be performed before installation of this module. The first test is done by default and can be run with:

perl Makefile.PL
make all
make test

The second set of tests is what is called author tests and is optionally run with:

perl Makefile.PL
make all
prove -bl xt

In both sets setting the environment variable TEMP_DIRS_KEEP to 1 will keep all temporary files created so that inspection of output files is possible. By default all temporary files created during testing are erased on test's exit.

MODELS

The OpenCV/WeChat QR code detector algorithm uses CNN (Convolutional Neural Networks) which are required to be trained first with data containing example QR code images. The detector kindly provided by OpenCV/WeChat already contains trained models (see "LICENSE AND COPYRIGHT" for license) which are also contained in and distributed with this module. These pre-trained models are installed as part of this module, along with everything else. You do not need to download them manually.

The pre-trained models can be found here where it also shows their MD5 signatures. Their total size is about 1 MB.

The models directory must contain four files: detect.caffemodel, detect.prototxt, sr.caffemodel and sr.prototxt.

Use modelsdir() to get the location of the installed models.

IMPLEMENTATION DETAILS

This code demonstrates how to call OpenCV (modern OpenCV v4) C++ methods using the technique suggested by Botje @ #perl in order to avoid all the function, macro, data structures name clashes between Perl and OpenCV (for example seed(), do_open(), do_close() and most notably struct cv and namespace cv in Perl and OpenCV respectively).

The trick suggested is to put all the OpenCV-calling code in a separate C++ file and provide high-level functions to be called by XS. So that the XS code does not see any OpenCV header files.

Makefile.PL will happily compile any .c and/or .cpp files found in the dir it resides by placing OBJECT => '$(O_FILES)' in %WriteMakefileArgs. And will have no problems with specifying also these:

CC      => 'g++',
LD      => 'g++',
XSOPT   => '-C++',

With one caveat, g++ compiler will mangle the names of the functions when placing them in the object files. And that will cause XSLoader to report missing and undefined symbols.

The cure to this is to wrap any function you want to remain unmangled between these two blocks:

#ifdef __cplusplus
extern "C" {
#endif

and

#ifdef __cplusplus
} //extern "C" {
#endif

This only need happen in the header file: wechat_qr_decode_lib.hpp and in the XS file where the Perl headers are included.

CAVEATS

Checking for whether local OpenCV installation has highgui support is currently very lame. It tries to detect it with three methods (see find_if_opencv_highgui_is_supported() in Makefile.PL for the implementation)

Note that DynaLoader (or FFI::CheckLib which uses it) can search for symbols in any library (e.g. highgui library should contain function imshow() in libopencv_highgui or libopencv_world). This would have been the most straight-forward way but alas, these are C++ libraries and the contained function names are mangled to weird function names like:

ZN2cv3viz6imshowERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_11_InputArrayERKNS_5Size_IiEE()

There's an imshow string in there but without a regex symbol-name search the symbol can not be detected. Currently, DynaLoader (which is called by FFI::CheckLib) does not provide a regular expression symbol name matching, only exact.

AUTHOR

Andreas Hadjiprocopis, <bliako at cpan.org>

BUGS

Please report any bugs or feature requests to bug-image-decodeqr-wechat at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Image-DecodeQR-WeChat. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc Image::DecodeQR::WeChat

You can also look for information at:

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

This software is Copyright (c) 2022 by Andreas Hadjiprocopis.

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)

The OpenCV/WeChat CNN trained models redistributed with this module are licensed under

The Apache License Version 2.0

See the full licence here.

HUGS

!Almaz!