AI-Based Sports Production with Intel Technology

Keemotion, an independent software vendor (ISV) and member of the Intel® AI Builders program, provides broadcasters, leagues, and sports teams with an automated production solution for live streaming indoor athletic events and practices. At the touch of a button, Keemotion creates AI enhanced video that offers a one-of-a-kind solution for immediate analysis of athlete and team performance. Their system is used by the NBA’s Golden State Warriors, as well as other collegiate and professional sports teams, providing access to consistent and reliable data on player performance – without the effort of filming games and practices and logging film.

Figure 1: Keemotion’s iPad and iPhone applications allow users to review plays in real time.

Figure 1: Keemotion’s iPad and iPhone applications allow users to review plays in real time.

How Keemotion Works

The Keemotion system uses video stitching technology combined with deep learning models to create video streams. Keemotion installs an Intel® Xeon® Scalable processor-based server in the arena and connects it to a proprietary three-camera rig to capture a panoramic view of the playing field, scoreboard, and surroundings. Keemotion’s algorithms create a seamless view of the field, selecting the most exciting subset to broadcast based on deep learning models.

One of the challenges that Keemotion faces is that the system must deliver the broadcast in as close to real time as possible. In use cases involving actual gameplay, the application needs to deliver the broadcast to sports platforms with minimum latency – ideally, below three seconds while continuing to run Keemotion’s machine learning (ML) algorithms. This makes software optimization essential.

Figure 2: Keemotion tracks all the action on the court and uses their video stitching algorithm to deliver a video to the end user in under three seconds.

Figure 2: Keemotion tracks all the action on the court and uses their video stitching algorithm to deliver a video to the end user in under three seconds.

Rising to the Challenge

One of the first functions Intel and Keemotion optimized was its video path, by taking advantage of Intel® Advanced Vector Extensions 512 (Intel® AVX-512) on systems powered by an Intel Xeon Scalable processor, which noticeably improved the performance of their video stitching algorithms.

To achieve an autonomous broadcast, Keemotion stitches together output of the cameras on the fly and creates a 180 degree view of the arena. They then select a “viewport” (a rectangular frame of 1920×1080 pixel resolution) to track the action. Their AI-enabled algorithm tracks the players and adjusts the center of the viewport (so the action seems more natural) before it’s streamed.

Thanks to a system powered by an Intel Xeon Scalable processor Keemotion is able to stitch court images, run their players detection AI model and calculate the optimal framing (viewport) in real time while achieving their desired latency requirement of less than three seconds

This optimization also allowed Keemotion to utilize a more advanced ML algorithm for player classification improving the system’s decision making and therefore increasing broadcast quality. These performance optimizations will soon allow the application to track action in other fast-paced sports such as soccer and hockey.

Figure 3: Keemotion’s three camera setup captures the whole court at above 4k resolutions.

Figure 3: Keemotion’s three camera setup captures the whole court at above 4k resolutions.

How Keemotion Applies Intel® AI Technologies

Keemotion’s main project was trained on the Intel® AI DevCloud, a cluster of Intel Xeon Scalable Processors and Intel-optimized software available exclusively to Intel AI Builders program members that allowed Keemotion to train, verify and fine-tune multiple models in parallel. Timing is critical for the business, and this training method will enable Keemotion to meet its deadlines. Alexandre Bustamante, Co-Founder and Head of Product for Keemotion, says “Intel® AI allowed us to unlock possibilities on image analysis that we could not before with traditional methods.” Capturing broadcast quality video footage typically requires truck-based camera rigs that must be set up each time, at significantly higher cost than the Keemotion solution. Keemotion now has three solution offerings powered by Intel technology:

To learn more, visit: www.keemotion.com or https://twitter.com/keemotion

Notices and Disclaimers: