Adaptive video streaming over HTTP, such as the MPEG-DASH standard, is now widely used by video service provides to stream their videos to users. But DASH and similar methods are known to suffer from two practical challenges: on the one hand, clients use fixed heuristics that limit their ability to generalize across network conditions, making the clients unable to efficiently predict variations in new networking environments, in turn leading to more buffering. On the other hand, the absence of collaboration among DASH clients leads to unfair bandwidth allocation, and typically pushes the system to an unbalanced equilibrium point. In this paper, we propose a server-side rate adaptation method that significantly improves the fairness of network bandwidth allocation among concurrent DASH users. We formulate the problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) model, and use Reinforcement Learning (RL) to train two neural networks to find an optimal solution to the fairness problem. Since our solution is implemented at the server side, it requires no modifications to the widely-installed DASH clients, making our solution very practical. We show that our proposed method outperforms the state-of-the-art schemes in terms of QoE-efficiency, QoE-fairness, and social welfare by as much as 16%, 21%, and 24% respectively.

HTTP adaptive streaming with chunked transfer encoding can be used to offer low-latency streaming without sacrificing the coding efficiency. While this allows a media segment to be generated and delivered at the same time, which is critical in reducing the latency, the conventional bitrate adaptation schemes make often grossly inaccurate bandwidth measurements due to the presence of idle periods between the chunks. These wrong measurements cause the streaming client to make bad adaptation decisions. To this end, we design ACTE, a new bitrate adaptation scheme that leverages the unique nature of chunk downloads. ACTE uses a sliding window to accurately measure the available bandwidth and an online linear adaptive filter to predict the bandwidth into the future. Results show that ACTE achieves 96% measurement accuracy, which translates to a 65% reduction in the number of stalls and a 49% increase in quality of experience on average compared to other schemes.

Virtual reality has been gaining popularity in recent years fueled by the proliferation of affordable consumer-grade devices such as Oculus Rift, HTC Vive, and Samsung VR. Amongst the various VR applications, 360° video streaming is currently one of the most popular ones. However, it poses a series of challenges to the serving content distribution systems. One challenge is the significantly increased bandwidth requirement for streaming such content in real time. Recent research has shown that only streaming the content that is in the user's (field-of-view) FoV in high quality can lead to strong bandwidth savings. This can be achieved by analyzing the viewers head orientation and movement based on sensor information. Alternatively, historic information from users that watched the content in the past can be considered to prefetch 360° video data in high quality assuming the viewer will direct the FoV to these areas. This paper presents a 360° video streaming system that transitions between sensor- and content-based predictive mechanisms. We evaluate the effects of our system on the Quality of Experience (QoE) of such a VR streaming system and show that the perceived quality can be increased between 50% and 80% compared to systems that only apply either one of the two approaches.

In 360-degree video streaming, Most solutions are based on tile-based streaming that divides videos into tiles and streams the high-quality tiles corresponding to the user's viewport areas. However, these methods cannot transmit different combinations of tile coding efficiently. In this paper, we experimented with streaming 360-degree videos using a motion-constrained tile set (MCTS) technique that allows encoding with constraining motion vectors such that each tile can be decoded and transmitted independently. Moreover, we have used a tile-based approach using a saliency map that integrates the information of human visual attention with the contents to deliver high-quality tiles to the region of interest (ROI). We encoded the 360-degree videos at various quality representations with MCTS techniques and assigned a tile quality representation using a saliency map predicted by the existing convolutional neural network (CNN) model. We proposed a novel heuristic algorithm to assign appropriate quality to the tiles on the centerline. Consequently, mixed quality videos based on the saliency map enable efficient streaming in 360-degree videos. Using the Salient360! dataset, the proposed method shows an improvement in terms of bandwidth with little loss of viewport image quality.

In this positioning paper, we propose Chord, a holistic multi-user VR system for untethered mobile devices over 802.11ac/ax Wi-Fi. Taking a cross-layer approach, Chord brings numerous innovations to the application-layer design and VR-aware wireless network optimizations. We present our design and its preliminary evaluation on commodity smartphones and 802.11ac Wi-Fi, to demonstrate the feasibility of Chord.

With the increase of HTTP-based adaptive video streaming over the Internet, multiple clients may compete for a shared bottleneck bandwidth, which brings some damage to the fairness and stability of Quality of Experience (QoE). This paper presents Steward, a system that enforces multi-client joint QoE optimization for bottleneck bandwidth sharing. Joint QoE optimization refers to improving QoE fairness among clients with various video devices and providing differentiated service for clients with different priorities. Steward deploys the adaptive bitrate (ABR) algorithm based on neural networks (NN) and reinforcement learning at the network edge. The ABR agent trains the NN model through experience and makes appropriate bitrate guidance for video chunks to be requested by clients sharing the same bottleneck bandwidth. We compare Steward with state-of-the-art algorithms under different network conditions. Compared with all considered algorithms and conditions, Steward reduces 30%~85% QoE unfairness under the premise of differentiated service.

Despite exciting prospects, the development of 360° videos is persistently hindered by the difficulty to stream them. To reduce the data rate, existing streaming strategies adapt the video rate to the user's Field of View (FoV), but the difficulty of predicting the FoV and persistent lack of bandwidth are important obstacles to achieve best experience. In this article we exploit the recent findings on human attention in VR to introduce a new additional degree of freedom for the streaming algorithm to leverage: Virtuall Walls (VWs) are designed to translate bandwidth limitation into a new type of impairment allowing to preserve the visual quality by subtly limiting the user's freedom in well-chosen periods. We carry out experiments with 18 users and confirm that, if the VW is positioned after the exploration phase in scenes with concentrated saliency, a substantial fraction of users seldom perceive it. With a double-stimulus approach, we show that, compared with a reference with no VW consuming the same amount of data, VW can improve the quality of experience. Simulation of different FoV-based streaming adaptations with and without VW show that VW enables reduction in stalls and increases quality in FoV.

Confronting with the sheer amount of Over-the-Top (OTT) videos, personalized recommendation is especially important for users to locate videos of interest. However, previous approaches seldom considered the influence of watching time when designing video recommendation algorithms. In this paper, we first conduct a detailed measurement study on a leading OTT video service provider in China and our results show that user view preferences are substantially influenced by watching time. Based on the above results, we further propose a personalized time-aware video recommendation algorithm called TAMF for OTT videos. The basic idea of our proposed TAMF algorithm is to utilize matrix factorization to unveil how watching time affects user view interests and cluster time slots with similar influence. In this way, we can collaboratively learn users' personal interests if their views belong to the same cluster, and precisely capture user view preferences with watching time. Finally, we also conduct extensive experiments using real traces to evaluate the performance of our algorithm, and the experimental results show that our proposed algorithm can improve video recommendation performance by 4.83% and 4.42% in terms of WMRR and WMAP respectively and significantly boost user engagement.

360° live video streaming is becoming increasingly popular. While providing viewers with enriched experience, 360° live video streaming is challenging to achieve since it requires a significantly higher bandwidth and a powerful computation infrastructure. A deeper understanding of this emerging system would benefit both viewers and system designers. Although prior works have extensively studied regular video streaming and 360° video on demand streaming, we for the first time investigate the performance of 360° live video streaming. We conduct a systematic measurement of YouTube's 360° live video streaming using various metrics in multiple practical settings. Our key findings suggest that viewers are advised not to live stream 4K 360° video, even when dynamic adaptive streaming over HTTP (DASH) is enabled. Instead, 1080p 360° live video can be played smoothly. However, the extremely large one-way video delay makes it only feasible for delay-tolerant broadcasting applications rather than real-time interactive applications. More importantly, we have concluded from our results that the primary design weakness of current systems lies in inefficient server processing, non-optimal rate adaptation, and conservative buffer management. Our research insight will help to build a clear understanding of today's 360° live video streaming and lay a foundation for future research on this emerging yet relatively unexplored area.

With the growing demand for crowdsourced live streaming (CLS), how to schedule the large-scale dynamic viewers effectively among different Content Delivery Network (CDN) providers has become one of the most significant challenges for CLS platforms. Although abundant algorithms have been proposed in recent years, they suffer from a critical limitation: due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we propose LTS (Learn to schedule), a deep reinforcement learning (DRL) based scheduling approach that can dynamically adapt to the variation of both viewer traffics and CDN performance. After the extensive evaluation the real data from a leading CLS platform in China, we demonstrate that LTS improves the average quality of experience (QoE) over state-of-the-art approach by 8.71%-15.63%.

The growing demand for video processing and the advantages in scalability and cost reduction brought by the emerging serverless computing have attracted significant attention in serverless computing powered video processing. However, how to implement and configure serverless functions to optimize the performance and cost of video processing applications remains unclear. In this paper, we explore the configuration and implementation schemes of typical video processing functions deployed to the serverless platforms and quantify their influence on the execution duration and monetary cost from a developer's perspective. Our measurement reveals that memory configuration is non-trivial. Dynamic profiling of workloads is necessary to find the best memory configuration. Moreover, compared with calling external video processing APIs, implementing these services locally in serverless functions can be competitive. We also find that the performance of video processing applications could be affected by the underlying infrastructure. Our work provides guidelines for further function-level optimization and complements the existing measurement studies for both serverless computing and video processing.

Mobile augmented reality (MAR) augments a real-world environment (probably surrounding or close to the mobile user) by computer-generated perceptual information. Utilizing the emerging edge computing paradigm in MAR systems can reduce the power consumption and computation load for the mobile devices and improve responsiveness of the MAR service. Different from existing studies that mainly explored how to better enable the MAR services utilizing edge computing resources, our focus is to optimize the video generation stage of the edge-based MAR services-efficiently using the available edge computing resources to render and encode the augmented reality as video streams to the mobile clients. Specifically, for multi-party AR applications, we identify the advantages and disadvantages of two encoding schemes, namely colocated encoding and spilt encoding, and examine the trade-off between performance and scalability when the rendering and encoding tasks are colocated or split. Towards optimally placing AR video rendering and encoding in the edge, we formulate and solve the rendering and encoding task assignment problem for multi-party edge-based MAR services to maximize the QoS for the users and the edge computing efficiency. The proposed task assignment scheme is proved to be superior through extensive trace-driven simulations and experiments on our prototype system.