RandAugment with Keypoints

Unofficial RandAugment[1] implementation for image and keypoint augmentation.

Motivation

Why RandAugment? RandAugment achieves state-of-the-art performance with a greatly reduced augmentation parameter search space. It's received additional attention due to its use in self-supervised learning models such as Unsupervised Data Augmentation(Xie et al.) and FixMatch(Sohn et al.).

Why this package? This package makes three contributions:

1. Provides an easily extendable framework in order to explore different magnitudes, policies, and augmentation operations.
2. Abstracts an "Augmentation Plan" that consists of magnitude, operations, and directions which can then be applied to both images and keypoints.
3. Provides additional support for consistency training by normalizing keypoint estimates produced from augmented images (see below).

Support for Consistency Training: Consistency Training has been used to attain state-of-the-art results on image classification problems [2, 3]. One challenge in adapting the classification technique proposed in [1] to handle keypoints is that the augmentations can cause the keypoints to become misaligned. In order to deal with this, we implement a RandAugment.apply_keypoints_inv, which takes keypoint predictions from augmented images and normalizes them so that they can be compared between augmentations.

Install

python setup.py install

Basic Usage

Using RandAugment Keypoints is simple. First use plan_augment in order to create (magnitude, operation, directions) and then apply them using apply_image or apply_keypoints.

Example:

import augkey

image = ... # your PIL.Image
randaug = augkey.RandAugment()
plan = randaug.plan_augment() 
augmented_image = randaug.apply_image(image, *plan)

References

1. Cubuk, Ekin D., Barret Zoph, Jonathon Shlens, and Quoc V. Le. “RandAugment: Practical Automated Data Augmentation with a Reduced Search Space.” ArXiv:1909.13719 [Cs], November 13, 2019. [http://arxiv.org/abs/1909.13719].

2. Sohn, Kihyuk, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. “FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence.” ArXiv:2001.07685 [Cs, Stat], November 25, 2020. [http://arxiv.org/abs/2001.07685].

3. Xie, Qizhe, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. “Unsupervised Data Augmentation for Consistency Training.” ArXiv:1904.12848 [Cs, Stat], November 5, 2020. [http://arxiv.org/abs/1904.12848].

Built With

• numpy
• pil
• python