ID |
原文 |
译文 |
873 |
响应性和稳定性一直是流式计算中两个至关重要的问题, |
Responsiveness and stability have always been two important problems in stream computing. |
874 |
而流计算系统在过载时常常表现出数据计算延迟增加和拓扑不稳定的现象,无法适应数据负载的动态变化。 |
However, asthe scale of data being processed in real-time has increased, along with an increase in the data processing latency and topolo-gy instability of stream computing, many limitations of stream processing system have become apparent. |
875 |
针对这一问题本文研究提出了一种基于动态拓扑的流计算性能优化方法,主要包括:(1)动态逐级反压:拓扑中的任务可以根据当前自身负载情况,动态调整上游向其发送数据的速率。 |
Aiming at theseproblems, we present a performance optimization method based on dynamic topology for stream computing:(1)Dynamicstep-by-step backpressure:the task in the topology can dynamically adjust the rate of upstream data transmission according to the current load. |
876 |
(2)无状态拓扑数据重放:拓扑不维持数据的计算状态,尽可能地实现数据容错。 |
(2)Stateless topology data replay: topology can achieve data fault tolerance autonomously without maintai-ning the calculation of data state. |
877 |
(3)自适应拓扑替换:在拓扑不暂停的情况下对任务并发度进行自发调整。 |
(3)Adaptive topology replacement: no need for topology to suspend, the system can adjustthe task concurrency spontaneously. |
878 |
(4)延迟持久化队列:拓扑中对磁盘的 IO 读写被延迟到数据处理之外,减缓 IO 高频阻塞对流计算系统的影响。 |
(4)Delayed persistent queue:it delays the IO reading and writing in the disk out of thedata processing, which mitigates the impact of IO high-frequency blocking in stream computing system. |
879 |
本文在 Apache Storm 中实现了以上四种方案, |
In this paper, thefour methods are implemented in Apache Storm. |
880 |
性能测试结果表明优化后的流计算系统与 Storm 默认实现相比,不仅增强了大数据动态匹配能力,而且在最优情况下改善了 17% 的吞吐量,并提升了约 20% 的数据处理速度。 |
The experimental results show that the optimized system not only enhancesthe dynamic matching capability of big data, but also achieves 17% higher throughput and 20% better data processing speedin the best case. |
881 |
率失真优化技术能有效地提升编码器的压缩性能,是视频编码领域重要的研究内容。 |
Rate-distortion optimization (RDO)can effectively improve the compression performance for encoders, which is an important topic in video coding. |
882 |
本文综合论述了基于拉格朗日乘子的率失真优化方法,从独立率失真优化、依赖率失真优化和码率控制中的比特分配等几方面归纳和分析最新研究进展。 |
In this paper, the Lagrange multiplier based RDO method is reviewed compre-hensively from several aspects including independent RDO, dependent RDO, bit allocation for rate control, etc. |