Massively Scalable Processing
Real-time processing systems designed to efficiently process large volumes of data in a distributed, massively scalable manner are known as massively scalable processing. Cloud-native solutions and distributed computing frameworks such as Hadoop and Spark are examples of such systems.
Features of MSP
Horizontal scalability Increasing the number of nodes (machines) to spread processing and storage over several systems is known as horizontal scalability.
Parallelism Dividing work into manageable portions that are handled concurrently by several nodes.
Fault tolerance Systems can gracefully bounce back from node outages or hardware malfunctions.
Scalability Distributed data storage allows for scalability of data access by distributing data among several nodes.
Dynamic Resource Allocation Allocating resources automatically in response to demand and load is known as dynamic resource allocation.
Use Case
Making use of scalable processing frameworks for big data analytics, real-time data processing, and ETL pipelines.
Massively Parallel Processing
Systems that are performing massively parallel large-scale processing utilizing multiple processors are known as massively parallel processing.
This approach is widely used in big data and analytics to handle massive datasets.
Features of MPP
Parallelism Several processors work on various aspects of a task at the same time.
Data partitioning Data partitioning is the process of dividing data into portions that are dispersed among nodes and handled separately.
Architecture of Shared Nothing
Every node has its own independent storage, memory, and CPU. Therefore, there is no resource contention, and it improves the scalability and fault tolerance.
Query parallelism SQL queries are divided and run concurrently on several nodes.
Data Locality To reduce data travel, computations are carried out on the nodes where the data is stored.
Use Case:
MPP architectures are used by database systems like Teradata, Snowflake, and Amazon Redshift to parallelize and spread queries across several nodes, allowing for quick query execution on large datasets.
Source link
lol
