How to Use AppVeyor to Build and Deploy Python Wheels from C/C++ Code

AppVeyor is a continuous integration (CI) service used to automatically build code projects and deploy relevant artifacts. It provides build environments for Windows, Linux, and macOS. In this article, I will share how to use AppVeyor to build and deploy Python Wheels ( Windows edition) from C/C++ code.

Creating Python Wheels

It takes two steps to build a Python wheel from CPython code:

1. Build *.pyd extension file from C/C++ code.
2. Pack *.pyd and dependent *.dll files into a wheel file.

For the past few years, I’ve been maintaining the source code of Python barcode extension based on Dynamsoft Barcode Reader C/C++ SDK. Let’s do some changes based on the repository.

The *.pyd Python module

Get the source code:

git clone https://github.com/yushulx/python.git

Install DynamsoftBarcodeReader7.2.1.exe. Copy *.dll files from Dynamsoft\Barcode Reader 7.2.1\Components\C_C++\Redist\x64 to bin folder and copy DBRx64.lib from Dynamsoft\Barcode Reader 7.2.1\Components\C_C++\Lib\DBRx64.lib to lib folder. Create an empty folder named wheel.

Open src\setup.py to set the link directories:

elif sys.platform == "win32":
    # Windows
    dbr_lib_name = 'DBRx64'
    dbr_lib_dir = r'..\lib'
    dbr_dll = r'..\bin'

Build the Python barcode module:

cd src
python setup.py build

The *.whl file

Create a folder named dbr under the wheel folder. Add a setup.py file:

The data_info contains the *.dll and *.whl files.

Create __init__.py under the dbr folder:

from .dbr import *

Copy *.pyd and *.dll files to the dbr folder from the src folder:

copy ..\bin\*.* ..\wheel\dbr\
cd build\lib.win-*\
copy *.* ..\..\..\wheel\dbr\

Build the wheel package under the wheel folder:

python setup.py bdist_wheel

Configuring YAML file for AppVeyor Build Service

Login https://ci.appveyor.com/ with your GitHub account.

Click Projects to import the target repository:

To trigger the build, create an appveyor.yml file under the project root directory.

Set the branch to build:

branches:
  only:
    - wheel

Add the Python environments:

environment:
  matrix:
    - APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
  PYTHON: "C:/Python27-x64"
    - PYTHON: "C:/Python35-x64"
    - PYTHON: "C:/Python36-x64"
    - PYTHON: "C:/Python37-x64"
    - PYTHON: "C:/Python38-x64"

Why do I use APPVEYOR_BUILD_WORKER_IMAGE for Python 2.7? When building the source code with the default environment, I got the following error message:

Traceback (most recent call last):
File "setup.py", line 63, in <module>
cmdclass={'install': CustomInstall}
File "C:\Python27-x64\lib\distutils\core.py", line 151, in setup
dist.run_commands()
File "C:\Python27-x64\lib\distutils\dist.py", line 953, in run_commands
self.run_command(cmd)
File "C:\Python27-x64\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "C:\Python27-x64\lib\distutils\command\build.py", line 127, in run
self.run_command(cmd_name)
File "C:\Python27-x64\lib\distutils\cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "C:\Python27-x64\lib\distutils\dist.py", line 972, in run_command
cmd_obj.run()
File "C:\Python27-x64\lib\distutils\command\build_ext.py", line 340, in run
self.build_extensions()
File "C:\Python27-x64\lib\distutils\command\build_ext.py", line 449, in build_extensions
self.build_extension(ext)
File "C:\Python27-x64\lib\distutils\command\build_ext.py", line 499, in build_extension
depends=ext.depends)
File "C:\Python27-x64\lib\distutils\msvc9compiler.py", line 473, in compile
self.initialize()
File "C:\Python27-x64\lib\distutils\msvc9compiler.py", line 383, in initialize
vc_env = query_vcvarsall(VERSION, plat_spec)
File "C:\Python27-x64\lib\distutils\msvc9compiler.py", line 299, in query_vcvarsall
raise ValueError(str(list(result.keys())))
ValueError: [u'path']
Command exited with code 1

The issue seems caused by Visual Studio 2008. According to the official documentation — Windows images software, Visual C++ 2008 Express is not installed in the Visual Studio 2019 image. So I can use the Visual Studio 2019 image and set Visual Studio 2015 as the build tool to make it work:

install:
- SET VS90COMNTOOLS=%VS140COMNTOOLS%

Add the build script:

build_script:
  - cmd: |
    "%PYTHON%/python.exe" -m pip install --upgrade pip
    "%PYTHON%/python.exe" -m pip install --upgrade setuptools wheel     numpy
  cd src
  "%PYTHON%/python.exe" setup.py build
  copy ..\bin\*.* ..\wheel\dbr\
  cd build\lib.win-*\
  copy *.* ..\..\..\wheel\dbr\
  cd ..\..\..\wheel\
  "%PYTHON%/python.exe" setup.py bdist_wheel

Set the artifacts:

artifacts:
  - path: wheel\dist\*.whl
    name: wheels

Add the scripts for PyPi deployment:

deploy_script:
  - ps: |
  if($env:APPVEYOR_REPO_TAG -eq 'true') {
  Write-Output ("Deploying " + $env:APPVEYOR_REPO_TAG_NAME + " to   PyPI...")
  &"${Env:PYTHON}/python.exe" -m pip install twine
  &"${Env:PYTHON}/python.exe" -m twine upload -u ${Env:USER} -p   ${Env:PASS} --skip-existing dist/*.whl
  } else {
  Write-Output "No tag for deployment"
  }

When adding tags to the GitHub repository, the APPVEYOR_REPO_TAG value will be ‘true’. You can encrypt the username and password of the PyPi account by clicking Account > Encrypt YAML.

When building successfully, the *.whl files for Python 2.7, 3.5, 3.6, 3.7 and 3.8 will be uploaded to the PyPi website.

Reference

• https://www.appveyor.com/docs/deployment/
• https://github.com/skvark/opencv-python/blob/master/appveyor.yml

Source Code

https://github.com/yushulx/python/tree/wheel

Originally published at https://www.codepool.biz on December 7, 2019.