ITERATIVE METHODS FOR THE SOLUTION OF LINEAR SYSTEM OF EQUATIONS

ABSTRACT Linear systems of equations play a central role in various fields, from engineering and physicstocomputer science and finance. These systems are often solved using numerical methods, whichare broadly categorized into direct methods (such as Gaussian elimination) and iterative methods(such as Jacobi, Gauss-Seidel, and Conjugate Gradient methods). Direct methods yieldexact solutions but are computationally expensive for large systems. Iterative methods, on theotherhand, provide a sequence of approximations that converge to the true solution. These methodsare particularly advantageous for large, sparse systems, where storing and manipulatingafull matrix is impractical. This project talks about the fundamental principles of iterative methods, discussing their advantages, limitations, and applications. We provided mathematical formulations, explain convergence criteria, and illustrate their implementation with numerical examples. In the course of this project, we investigated the performance of four popular iterative methods–Jacobi Method, Gauss-Seidel Method, Successive Over-Relaxation (SOR), and ConjugateGradient (CG) Method in solving a system of linear equations represented by a 5x5 matrix. Wediscovered that all methods converge to the solution, but at different rates and withvaryingcomputational efficiency. Jacobi Method converged slowly due to its reliance on values fromtheprevious iteration without real-time updates. Gauss-Seidel Method improved on Jacobi byincorporating updated values within each iteration, resulting in faster convergence. SORMethod, with an appropriate relaxation factor (ω), further accelerated convergence beyond Gauss-Seidel, especially when ω was optimally chosen (e.g., between 1 and 2). Conjugate Gradient Methodexhibited the fastest convergence among all methods, requiring fewer iterations and achievinghigher accuracy in a shorter time. This is expected, as Conjugate Gradient Method is specificallydesigned for symmetric positive definite matrices, which the system was modified toensure. Thus, it is confirmed that the Conjugate Gradient Method is the most efficient for solvinglarge, sparse, and symmetric positive definite systems, while Gauss-Seidel and SORoffer moderateimprovements over the basic Jacobi approach.