Tag Archives: koutsoukos

Linearize, predict and place: minimizing the makespan for edge-based stream processing of directed acyclic graphs

Khare, S., Sun, H., Gascon-Samson, J., Zhang, K., Gokhale, A., Barve, Y., Bhattacharjee, A. and Koutsoukos, X. (2019). Linearize, predict and place: minimizing the makespan for edge-based stream processing of directed acyclic graphs. Symposium of Edge Computing (SEC) 2019
[Preprint]

Abstract: Many IoT applications found in cyber-physical systems, such as smart grids, must take control actions in response to critical events, such as supply-demand mismatch, which requires low-latency processing of streaming data for rapid event detection and anomaly remediation. These streaming applications generally take the form of directed acyclic graphs (DAGs), where vertices represent operators and edges represent the flow of data between these operators. Edge computing has recently attracted significant attention as a means to readily meet the requirements of latency-critical IoT applications due to its ability to provide low-latency processing near the source of data. To accrue the benefits of edge computing, the constituent operators of these applications must be placed in a manner that intelligently trades-off inter-operator communication costs with the cost of interference incurred due to co-location of operators on the same resource-constrained edge devices. To address these challenges and to substantially simplify the placement problem for DAGs of arbitrary sizes and topologies, we present an algorithm that first transforms any arbitrary stream processing DAG into an approximate set of linear chains. Subsequently, a data-driven latency prediction model for co-located linear chains is used to inform the placement of operators such that the makespan, defined as the maximum latency of all paths in the DAG, is minimized. We empirically evaluate our algorithm using a variety of DAG placement scenarios on a Beagle Bone cluster, which is representative of an edge computing environment.

Scalable Edge Computing for Low Latency Data Dissemination in Topic-Based Publish/Subscribe

Khare, S., Sun, H., Zhang, K., Gascon-Samson, J., Gokhale, A., Koutsoukos, K., Abdelaziz H. (2018) Scalable Edge Computing for Low Latency Data Dissemination in Topic-Based Publish/Subscribe, 2018 IEEE/ACM Symposium on Edge Computing (SEC 2018), Seattle, WA, USA
[Preprint] [Presentation Slides]

Abstract: Advances in Internet of Things (IoT) give rise to a variety of latency-sensitive, closed-loop applications that reside at the edge. These applications often involve a large number of sensors that generate volumes of data, which must be processed and disseminated in real-time to potentially a large number of entities for actuation, thereby forming a closed-loop, publish-process-subscribe system. To meet the response time requirements of such applications, this paper presents techniques to realize a scalable, fog/edge-based broker architecture that balances data publication and processing loads for topic-based, publish-process-subscribe systems operating at the edge, and assures the Quality-of-Service (QoS), specified as the 90th percentile latency, on a per-topic basis. The key contributions include: (a) a sensitivity analysis to understand the impact of features such as publishing rate, number of subscribers, per-sample processing interval and background load on a topic’s performance; (b) a latency prediction model for a set of co-located topics, which is then used for the latency-aware placement of topics on brokers; and (c) an optimization problem formulation for k-topic co-location to minimize the number of brokers while meeting each topic’s QoS requirement. Here, k denotes the maximum number of topics that can be placed on a broker. We show that the problem is NP-hard for k >=3 and present three load balancing heuristics. Empirical results are presented to validate the latency prediction model and to evaluate the performance of the proposed heuristics.

Poster Abstract: Ensuring Low-Latency and Scalable Data Dissemination for Smart-City Applications

Khare, S., Sun, H., Zhang, K., Gascon-Samson, J., Gokhale, A., Koutsoukos, K. (2018) Poster Abstract: Ensuring Low-Latency and Scalable Data Dissemination for Smart-City Applications, IoTDI 2018, Orlando, USA

Abstract: Low latency and scalable data dissemination is a critical requirement for many IoT applications, e.g., smart city applications, which are often built over a publish/subscribe communication paradigm. Ensuring low latency requires effective load balancing of the publish/subscribe topics across the different publishers and subscribers. To that end we present ongoing work on a data-driven approach to learning a latency-aware model of IoT broker loads, and in turn using it to determine broker replication, and balancing topics across them.