Scalability is among the most crucial elements in modern pipelines for data processing. With the exponential growth of data produced by applications devices, devices, and users organisations must create pipelines that can handle the growing volume, velocity and diversity of data without sacrificing the performance. A pipeline that is scalable ensures that when workloads increase the system will...