№2, 2025

OPTIMIZATION OF ACCESS TO STATIC DATA IN DISTRIBUTED SYSTEMS: A KUBERNETES-BASED SOLUTION WITH POSTGRESQL AND DJANGO

Nail Mammadov, Gulshan Umud Mammadova, Tunar Babalı

Static data is a crucial component in distributed systems, ensuring the seamless operation of various components. However, achieving reliable and high-frequency access to such data poses challenges due to its heterogeneous structure. This paper introduces a Kubernetes-orchestrated Static Data Database that integrates advanced technologies and best practices to address these challenges effectively. The proposed system leverages the Django framework and PostgreSQL database, optimized with advanced features such as JSONB indexing and metadata flexibility. These technologies are not only widely adopted in the industry for their proven scalability and efficiency but also incorporate cutting-edge academic innovations, such as JSONB-based metadata management and modular database schemas, to enhance flexibility in diverse use cases. Data payloads are stored in a POSIX-compliant distributed file system to ensure robustness. The service is containerized and deployed using Helm on the Kubernetes platform, with OKD serving as the deployment environment to achieve scalability and operational efficiency. This deployment model reflects industrial standards for cloud-native applications while demonstrating the practical applicability of academic research on container orchestration and resource optimization. Through extensive testing, the system demonstrated significant scalability, reliability, and performance improvements under high-demand scenarios. The testing process was designed to mirror real-world industrial workloads, such as high-frequency data access and concurrent queries while validating academic hypotheses on system behavior under extreme conditions. The results validate its potential to meet the growing needs of modern distributed systems, offering a scalable and future-ready solution firmly grounded in academic research and industrial practice (pp.56-68).

Keywords: PostgreSQL, Django, Kubernetes-Orchestrated, Amazon EKS, Nginx.
References
  • Abbasi, M., Bernardo, M. V., Váz, P., Silva, J., & Martins, P. (2024a). Adaptive and scalable database management with machine learning integration: A PostgreSQL case study. Information, 15(9), 574. https://doi.org/10.3390/info15090574
  • Abbasi, M., Bernardo, M. V., Váz, P., Silva, J., & Martins, P. (2024b). Revisiting database indexing for parallel and accelerated computing: A comprehensive study and novel approaches. Information, 15(8), 429. https://doi.org/10.3390/info15080429
  • Amer, M. A., Chervenak, A., & Chen, W. (2012). Improving scientific workflow performance using policy-based data placement. 2012 IEEE International Symposium on Policies for Distributed Systems and Networks (POLICY). https://doi.org/10.1109/POLICY.2012.8
  • Augustyn, D. R., Wyciślik, Ł., & Sojka, M. (2024). Tuning a Kubernetes horizontal pod autoscaler for meeting performance and load demands in cloud deployments. Applied Sciences, 14(2), 646. https://doi.org/10.3390/app14020646
  • AWS Documentation. (2022). Amazon Elastic Kubernetes Service (EKS) User Guide. Retrieved from https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html
  • Bakiras, S., Loukopoulos, T., Papadias, D., & Ahmad, I. (2005). Adaptive schemes for distributed web caching. Journal of Parallel and Distributed Computing, 65(12), 1483–1496. https://doi.org/10.1016/j.jpdc.2005.05.020
  • Bhavsar, S., Agrawal, A., Ropalkar, T., & Kamdi, P. (2023). Kubernetes cluster disaster recovery using AWS. In 2023 7th IEEE International Conference on Computing, Communication, Control and Automation (ICCUBEA). https://doi.org/10.1109/ICCUBEA58933.2023.10391973
  • Carrión, M. D. C. (2022). Kubernetes as a standard container orchestrator - A bibliometric analysis. Journal of Grid Computing, 20(4). https://doi.org/10.1007/s10723-022-09629-8
  • Curino, C., Jones, E. P. C., Madden, S., & Balakrishnan, H. (2011). Workload-aware database monitoring and consolidation. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (SIGMOD '11) (pp. 313–324). https://doi.org/10.1145/1989323.1989357
  • Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107-113. https://doi.org/10.1145/1327452.1327492
  • Ganesan, A., Alagappan, R., Arpaci-Dusseau, A. C., & Arpaci-Dusseau, R. H. (2017). Redundancy does not imply fault tolerance: Analysis of distributed storage reactions to file-system faults. ACM Transactions on Storage (TOS), 13(3), Article 20, 1–33. https://doi.org/10.1145/3125497
  • Graur, D., Müller, I., Proffitt, M., Fourny, G., Watts, G. T., & Alonso, G. (2023). Evaluating query languages and systems for high-energy physics data. Journal of Physics: Conference Series, 2438(1), 012034. https://doi.org/10.1088/1742-6596/2438/1/012034
  • HSF Collaboration. (2017). A Roadmap for HEP Software and Computing R&D for the 2020s. CERN White Paper. Retrieved from https://cds.cern.ch/record/2311808
  • Jani, Y. (2024). Unified monitoring for microservices: Implementing Prometheus and Grafana for scalable solutions. Journal of Artificial Intelligence, Machine Learning, and Data Science, 2(1), 1-5. https://doi.org/10.51219/JAIMLD/yash-jani/206
  • Kubernetes Documentation. (2023). Horizontal Pod Autoscaler (HPA) for Scalable Applications. Retrieved from https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
  • Lencha, A. A., Mitiku, A. B., & Woldemichael, A. T. (2024). Secure and modular data portal: Database system to manage broadly classified and large-scale data. Data Science Journal, 23(20), 1-17. https://doi.org/10.5334/dsj-2024-020
  • Mahida, A. (2023). Enhancing observability in distributed systems: A comprehensive review. Journal of Mathematical & Computer Applications, 2(3), 1–4. https://doi.org/10.47363/JMCA/2023(2)1
  • Makris, A., Tserpes, K., Spiliopoulos, G., & Zissis, D. (2021). MongoDB vs PostgreSQL: A comparative study on performance aspects. GeoInformatica, 25, 243–268. https://doi.org/10.1007/s10707-020-00407-w
  • Malviya, A., & Dwivedi, R. K. (2022). A comparative analysis of container orchestration tools in cloud computing. In 2022 9th IEEE International Conference on Computing for Sustainable Global Development (INDIACom) (pp. 23–25). https://doi.org/10.23919/INDIACom54597.2022.9763171
  • PostgreSQL Global Development Group. (2020). PostgreSQL Documentation: JSONB Data Type and Indexing. Retrieved from
  • https://www.postgresql.org/docs/current/datatype-json.html
  • Rabieyan, R., Yahyapour, R., & Jahnke, P. (2024). Optimization of containerized application deployment in virtualized environments: A novel mathematical framework for resource-efficient and energy-aware server infrastructure. The Journal of Supercomputing, 80, 22598–22630. https://doi.org/10.1007/s11227-024-06304-5
  • Saputra, M. Y. E., Noprianto, N., Arief, S. N., & Wijayaningrum, V. N. (2024). Real-time server monitoring and notification system with Prometheus, Grafana, and Telegram integration. 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS). https://doi.org/10.1109/ICETSIS61505.2024.10459488
  • Trần, M.-N., Vu Xuan, T., & Kim, Y. (2022). Proactive stateful fault-tolerant system for Kubernetes containerized services. IEEE Access, 10, 102181-102194. https://doi.org/10.1109/ACCESS.2022.3209257
  • Verreydt, S., Heydari Beni, E., Truyen, E., & Lagaisse, B. (2019). Leveraging Kubernetes for adaptive and cost-efficient resource management. In Proceedings of the 5th International Workshop on Container Technologies and Container Clouds (WOC '19) (pp. 37-42). https://doi.org/10.1145/3366615.3368357
  • Yadav, M. P., Raj, G., Akarte, H., & Yadav, D. K. (2020). Horizontal scaling for containerized applications using a hybrid approach. Ingénierie des systèmes d'information, 25(6), 709–718. https://doi.org/10.18280/isi.250601
  • Zhang, L., Pang, K., Xu, J., & Niu, B. (2022). JSON-based control model for SQL and NoSQL data conversion in hybrid cloud database. Journal of Cloud Computing: Advances, Systems and Applications, 11(23). https://doi.org/10.1186/s13677-022-00302-9