AERIS: ADAPTIVE EXECUTOR-SCOPED RESOURCE MANAGEMENT FOR PRIVACY-PRESERVING DATA PROCESSING IN APACHE SPARK
Keywords:
Apache spark, Privacy-preserving analytics, Resource management, Executor affinity, Encrypted spill, Differential privacyAbstract
Privacy-preserving Spark pipelines introduce executor-local costs that conventional resource policies do not model: cryptographic context reconstruction, enlarged protected working sets, and encrypted spill. This paper presents AERIS, an adaptive executor-scoped controller that couples task memory grants with soft affinity to reusable privacy state. AERIS operates only on coarse metadata—partition size, operator class, key domain, and executor load—and therefore does not inspect plaintext, keys, or differentially private outputs. Its cost estimator is calibrated by real local microbenchmarks of AES-GCM, HMAC-SHA256, bounded Laplace-noised aggregation, scrypt context setup, and encrypted temporary-file spill. We evaluate the design with an event-driven prototype aligned with Spark stage, task, barrier, and executor semantics. The prototype uses the real Wisconsin Diagnostic Breast Cancer dataset and deterministic bootstrap expansion to generate balanced, skewed, and bursty traces. Across ten paired seeds, AERIS reduces makespan by 9.3%, 16.6%, and 10.4% relative to fixed uniform executor budgets. In the two memory-stressed workloads it reduces encrypted spill volume by 62.7% and 64.4%. Against a stronger reactive load-aware baseline, AERIS is 1.5% faster on balanced traces, 0.6% slower on skewed traces, and 5.5% slower on bursty traces, while reducing encrypted spill by 74.4% and 81.0% and lowering p95 service time by 1.0%–3.7%. Queue-inclusive p95 completion latency worsens under skew because longest-first dispatch and state reuse delay short tasks; this limitation is reported rather than hidden. The results support executor-scoped privacy state as a useful scheduling dimension, while also showing that spill avoidance and completion-tail objectives must be balanced explicitly. The artifact is a measurement-backed, trace-driven prototype, not a claim of measurements from a patched Spark cluster.References
[1] Zaharia M, Chowdhury M, Franklin M J, et al. Spark: Cluster Computing with Working Sets. Proceedings of the 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud), 2010.
[2] Zaharia M. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2012: 15-28.
[3] Armbrust M. Spark SQL: Relational Data Processing in Spark. Proceedings of the ACM SIGMOD International Conference on Management of Data, 2015: 1383-1394. DOI: 10.1145/2723372.2742797.
[4] Armbrust M. Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark. Proceedings of the ACM SIGMOD International Conference on Management of Data, 2018: 601-613. DOI: 10.1145/3183713.3190664.
[5] Venkataraman S, Panda A, Ousterhout K, et al. Drizzle: Fast and Adaptable Stream Processing at Scale. Proceedings of the 26th ACM Symposium on Operating Systems Principles (SOSP), 2017: 374-389. DOI: 10.1145/3132747.3132750.
[6] Hindman B. Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2011: 295-308.
[7] Ghodsi A, Zaharia M, Hindman B, et al. Dominant Resource Fairness: Fair Allocation of Multiple Resource Types. Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2011: 323-336.
[8] Ousterhout K, Wendell P, Zaharia M, et al. Sparrow: Distributed, Low Latency Scheduling. Proceedings of the 24th ACM Symposium on Operating Systems Principles (SOSP), 2013: 69-84. DOI: 10.1145/2517349.2522716.
[9] Mao H, Schwarzkopf M, Venkatakrishnan S B, et al. Learning Scheduling Algorithms for Data Processing Clusters. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM), 2019: 270-288. DOI: 10.1145/3341302.3342080.
[10] Venkataraman S, Yang Z, Franklin M, et al. Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics. Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2016: 363-378.
[11] Shen Y, Ren X, Lu Y, et al. Rover: An Online Spark SQL Tuning Service via Generalized Transfer Learning. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023: 4800-4812. DOI: 10.1145/3580305.3599953.
[12] Zheng W. Opaque: An Oblivious and Encrypted Distributed Analytics Platform. Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017: 283-298.
[13] Schuster F. VC3: Trustworthy Data Analytics in the Cloud Using SGX. Proceedings of the IEEE Symposium on Security and Privacy, 2015: 38-54. DOI: 10.1109/SP.2015.10.
[14] Arnautov S. SCONE: Secure Linux Containers with Intel SGX. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016: 689-703.
[15] Hunt T. Ryoan: A Distributed Sandbox for Untrusted Computation on Secret Data. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016: 533-549.
[16] Papadimitriou A. Big Data Analytics over Encrypted Datasets with Seabed. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2016: 587-602.
[17] Popa R A, Redfield C M S, Zeldovich N, et al. CryptDB: Protecting Confidentiality with Encrypted Query Processing. Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP), 2011: 85-100. DOI: 10.1145/2043556.2043566.
[18] Tu S, Kaashoek M F, Madden S, et al. Processing Analytical Queries over Encrypted Data. Proceedings of the VLDB Endowment, 2013, 6(5): 289-300. DOI: 10.14778/2535573.2488336.
[19] Dwork C. Differential Privacy. In: Automata, Languages and Programming, LNCS 4052. Springer, 2006: 1-12. DOI: 10.1007/11787006_1.
[20] McSherry F. Privacy Integrated Queries: An Extensible Platform for Privacy-Preserving Data Analysis. Proceedings of the ACM SIGMOD International Conference on Management of Data, 2009: 19-30. DOI: 10.1145/1559845.1559850.
[21] Roy I, Setty S T V, Kilzer A, et al. Airavat: Security and Privacy for MapReduce. Proceedings of the 7th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2010: 297-312.
[22] Mohan P, Thakurta A, Shi E, et al. GUPT: Privacy Preserving Data Analysis Made Easy. Proceedings of the ACM SIGMOD International Conference on Management of Data, 2012: 349-360. DOI: 10.1145/2213836.2213876.
[23] Wilson R J, Zhang C Y, Lam W, et al. Differentially Private SQL with Bounded User Contribution. Proceedings on Privacy Enhancing Technologies, 2020, 2020(2): 230-250. DOI: 10.2478/popets-2020-0025.
[24] Johnson N M, Near J P, Hellerstein J M, et al. Chorus: A Programming Framework for Building Scalable Differential Privacy Mechanisms. Proceedings of the IEEE European Symposium on Security and Privacy, 2020: 535-551. DOI: 10.1109/EuroSP48549.2020.00041.
[25] Kotsogiannis I, Tao Y, He X, et al. PrivateSQL: A Differentially Private SQL Query Engine. Proceedings of the VLDB Endowment, 2019, 12(11): 1371-1384. DOI: 10.14778/3342263.3342274.
[26] Wolberg W H, Street W N, Mangasarian O L. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository, 1995. DOI: 10.24432/C5DW2B.
[27] Pedregosa F, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 2011, 12: 2825-2830.
[28] National Institute of Standards and Technology. Advanced Encryption Standard (AES). FIPS PUB 197-upd1, 2023. DOI: 10.6028/NIST.FIPS.197-upd1.
[29] Dworkin M. Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC. NIST Special Publication 800-38D, 2007. DOI: 10.6028/NIST.SP.800-38D.