Tencent AR researcher Liu Xiao
December 6, SIGGRAPH Asia 2016 is held in Macau, a considerable part of this year are related with the VR / AR. Today, in the Tencent You Tu lab, Tsinghua University - Tencent joint lab and Tencent AI Lab jointly held a sharing session, from the Tencent Liu Xiao for the first time introduced the Tencent "QAR open platform", an augmented reality platform .
Tencent engaged in AR research for more than three years of Liu Xiao told Lei Feng network (public: Lei Feng network), the platform has not yet launched, is expected to be released next year along with other products.
The following content from Liu Xiao's Keynote speech.
In recent years, the field of AR increasingly fire, especially in the areas of advertising and games. The current AR can be divided into two kinds: one is moving AR, the other is the head-mounted display. Because they use different sensors, usually its tracking technology is also different.
AR developers want to develop applications, the AR SDK is currently more popular Vuforia, ARToolkit, Metaio and Wikitude. The app developers encountered the problem is:
1, some powerful API for entry developers too difficult;
2, for the lightweight App is too heavy;
3, model rendering requires additional work;
4, the establishment of identification services require additional work;
5, some of the leading features are not included: SLAM, outdoor facial AR and so on.
QAR = Quick + AR, the goal is to build easy-to-use, lightweight, one-stop, multi-feature and free AR SDK, developers need only three lines of code can add their own App Ten module AR function.
On the "Quick" definition:
The live video demonstrates four demo titles: The Smurfs 2 and NARUTO 3D animated demo, The Avengers AR trailer demo, and an AR camera demo with real 3D models.
QAR Technology Overview
AR network server
First, we set up an AR web server for Image Queue and Model Management. User interaction with the AR database in two steps, first upload the Marker image to the server, and then the server will identify them, and adjust the AR model with matching, and then display. The whole process is very rapid, and supports multi-user and large user scale, each user has its own database.
For image and object retrieval, we used the CDVS (Compact Descriptors for Visual Search) algorithm.
Detection and tracking
The detection tracing is the core of AR technology. In the detection, we use the ORB detector, based on the general search using the bags of words method, using Homography for 2D images and PnP + RANSAC for 3D images. This allows us to support retrieval of less than 1000 targets on the mobile platform.
The following figure shows the technical implementation of the tracing scheme. We use the SLAM-based tracking technique, because we want to support both the target and the scene tracking.
Model conversion and rendering
In terms of model rendering, we used a lightweight engine based on Open GL ES, Android only 66kb, iOS 140kb. At the same time have a high rendering speed, in the low-end Android phone is less than 1.5ms.
Scene tracking: SLAM
For buildings, offices and other places, we do not need Marker point SLAM tracking technology. At the same time, we use the Kalman filter to combine the IMU and camera data to provide fast and consistent tracking.
QAR has not yet released, but there are already Tencent internal applications are very interested in this platform, the following share some of the application scenarios.
1, WeChat movie tickets;
2, RTX + Tencent Explorer;
3, QQ mobile browser;
5, WeChat open class;
6, WeChat sweep sweep;
7, mobile phone housekeeper;
8, the application of Po.
WeChat movie tickets, for example, the user can open the camera in the App aimed at posters, such as popular movie "days will be", after the 3D Jackie Chan will jump out, the user can and his friends to share a photo circle.
At present, we are working with many internal applications to explore the AR application scenarios, are cooperative projects include: RTX + Tencent Explorer, WeChat, QQ browser, mobile phone housekeeper and application Po.
QAR future direction
In addition to the existing target tracking feature, we will add three additional features in the QAR SDK: Face AR, Handy AR, and Scene AR.
In addition to App developers, we will also launch H5 open platform for web developers. Later, we hope to support a variety of headdrives, including HoloLens, Meta and so on.
Developers who want to understand QAR can access itWww.aropen.com.